-- Logs begin at Wed 2022-07-06 14:42:34 UTC, end at Wed 2022-07-06 15:44:34 UTC. -- Jul 06 15:10:53.785470 dv-cinder-848848-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 06 15:10:53.794112 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 6 15:10:53 2022] *** Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: nodename: dv-cinder-848848-4-iscsi Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: machine: x86_64 Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: clock source: unix Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: pcre jit disabled Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: detected number of CPU cores: 4 Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: current working directory: / Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: detected binary path: /usr/bin/uwsgi-core Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: your processes number limit is 47680 Jul 06 15:10:53.796940 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: your memory page size is 4096 bytes Jul 06 15:10:53.797615 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: detected max file descriptor number: 2048 Jul 06 15:10:53.797615 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: lock engine: pthread robust mutexes Jul 06 15:10:53.797615 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: thunder lock: enabled Jul 06 15:10:53.797615 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 06 15:10:53.797615 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 06 15:10:53.840852 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: Python main interpreter initialized at 0x56265be6b9c0 Jul 06 15:10:53.840852 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: python threads support enabled Jul 06 15:10:53.840852 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: your server socket listen backlog is limited to 100 connections Jul 06 15:10:53.840852 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: your mercy for graceful operations on workers is 90 seconds Jul 06 15:10:53.841153 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: mapped 403077 bytes (393 KB) for 2 cores Jul 06 15:10:53.841250 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: *** Operational MODE: preforking *** Jul 06 15:10:53.841521 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: *** uWSGI is running in multiple interpreter mode *** Jul 06 15:10:53.841521 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: spawned uWSGI master process (pid: 80121) Jul 06 15:10:53.841705 dv-cinder-848848-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 06 15:10:53.847293 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: spawned uWSGI worker 1 (pid: 80130, cores: 1) Jul 06 15:10:53.847293 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: spawned uWSGI worker 2 (pid: 80131, cores: 1) Jul 06 15:10:53.847293 dv-cinder-848848-4-iscsi devstack@keystone.service[80121]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 06 15:10:59.280408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 15:10:59.280408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jul 06 15:10:59.281183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 15:10:59.281183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jul 06 15:10:59.281318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 15:10:59.281318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jul 06 15:10:59.281318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 15:10:59.281318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jul 06 15:10:59.331174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 15:10:59.331337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 15:10:59.331416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 15:10:59.331501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 15:10:59.331622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] command line args: [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 15:10:59.331751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] command line args: [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 15:10:59.331918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 15:10:59.332004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 15:10:59.332123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ================================================================================ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 15:10:59.332200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ================================================================================ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 15:10:59.332401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.332491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.332655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_dir = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.332735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_dir = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.332857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.332933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_source = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_source = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] debug = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] debug = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.333980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.334988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_config_append = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_config_append = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_dir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_dir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.335972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.336907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.337870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.338980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.339081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.339945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] publish_errors = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] publish_errors = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.340921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] standard_threads = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] standard_threads = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_journal = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.341987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_json = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_journal = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_stderr = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_json = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_syslog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_stderr = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_syslog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.342987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 15:10:59.343060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.343915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.344903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.345606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.345692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.345861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.345938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.346953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.347967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.348889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.349906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.350880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.351882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352144 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.352878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.353983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.354964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.355937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.356920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.357857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.358982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.359972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.360057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.360212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.360290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.360481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.360563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.361843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.362974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.363961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.364932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.365965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.366918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.367880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.368849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.external = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.external = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.369986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.password = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.token = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.password = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.370986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.token = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.371972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.372935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.373999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.374977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375144 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.375954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.376969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.377942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.378989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.379913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.380961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.381928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.382896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.383939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.384955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.385901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.386860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.387877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.388981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.389945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.390985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.391747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.392973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.393949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.394914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.395881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.396873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.397991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.398958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.399950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.400994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.401955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.402947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.403924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.404881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.405893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.406863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.407027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.407102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.407301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.407383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.408789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.409872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.410858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.411850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.412859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.413974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.414959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.415951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.416994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.417928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.418984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.419994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.420948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.421905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 15:10:59.422847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 15:10:59.422957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 15:10:59.453356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.453586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.453672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.453823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.453906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.454053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.462851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.463069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.491393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.491630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.491802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.492044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.633135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.633638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.682883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.683127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.683522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.683754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.748481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.748715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.748945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.749839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.750062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.774399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.774644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.774764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.774918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.775922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.776065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.787756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.787896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.800830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.801955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.802036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.802172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.802252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.802395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 15:10:59.820276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.820484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.820569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 15:10:59.820701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 15:10:59.820823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.821031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 15:10:59.821261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.821341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.821471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 15:10:59.821552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.821750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.821831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.822022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.822101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.822287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.822550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.833813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.834824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.835106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.842986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.843475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.846358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.846641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.846805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.846925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.847978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.848879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.849071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.849160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.849343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.849615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.875602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.876845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.877744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.878029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.878467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.878747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.883375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.883623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.884029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.884114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.884320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.884608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.891373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.891666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.892818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.893101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.896770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.897063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.897353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.897597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.897692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.897891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.897972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.898988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.899287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.899562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.917601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.917888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.918366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.918654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.922766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.923579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.926464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.926766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.927198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.927518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.931938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.932762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.935570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.936415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.939792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.940099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.940387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.940635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.940716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.940939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.941033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.941218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.941298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.941532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.941822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.942107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.960865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.961148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.961420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.961686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.961778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.962023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.962293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.962560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.969648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.970526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.971426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.971707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.972274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.972557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.974912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.975198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.975509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.975798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.975879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.976103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.976191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.976388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.976673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.977011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.989451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.989738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.990023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.990321 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.990400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:10:59.990608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.990690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.990904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.991208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.991518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.991811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.992085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.992195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:10:59.992493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.010929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.011734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.014248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.015014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.018134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.018902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.019911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.020539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.021570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.021853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.022143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.022325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.022608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.022901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.029559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.030286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.031184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.031939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.034471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.034769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.035166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.035264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.035478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.035572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.035782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.036063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.044899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.045191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.045477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.045570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.045831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.046113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.053430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.053715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 15:11:00.054280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.054958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.055817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.056982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.057180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.057261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.057448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.057729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 15:11:00.096108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56265be6b9c0 pid: 80130 (default app) Jul 06 15:11:00.096719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56265be6b9c0 pid: 80131 (default app) Jul 06 15:11:00.099516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bf71-f328-4c35-898c-ebec2fe7c60b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:00.099820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bf71-f328-4c35-898c-ebec2fe7c60b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:00.100105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2bf71-f328-4c35-898c-ebec2fe7c60b None None] PATH_INFO: `/v3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:00.104351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Wed Jul 6 15:11:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:07.606876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:07.607426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:07.607426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:07.611752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:07.617189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:07.617597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:07.617755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:07.687061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: INFO dbcounter [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] Registered counter for database keystone Jul 06 15:11:07.708322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG oslo_db.sqlalchemy.engines [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80130) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 15:11:07.714464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writer thread running {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 15:11:07.889843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80130) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 15:11:07.890194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80130) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 15:11:07.944987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:07.946619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:08.066977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 06 15:11:08.236604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:08.306092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: INFO dbcounter [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] Registered counter for database keystone Jul 06 15:11:08.324948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG oslo_db.sqlalchemy.engines [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 15:11:08.331020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writer thread running {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 15:11:08.447630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204e358-e187-488a-8c76-3ebdbd939eb9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:08.451852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:08.452520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:08.452806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:08.453083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:08.461588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:08.512948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 15:11:08.513241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7284b-d82d-464d-a00a-31f5c7f2c789 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:08.523107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Jul 6 15:11:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 15:11:08.528968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:08.544012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ad9aa-46a9-4944-96da-301fb37484fa None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:08.548663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:08.549787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:08.550082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:08.550361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] PATH_INFO: `/v3/projects/b898c8312f89416b8b105c391af48831` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:08.551893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] RBAC: Authorizing `identity:get_project(project_id=b898c8312f89416b8b105c391af48831)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:08.605058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 15:11:08.605407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5867f-ac3c-4d99-9e08-8dd9348e22da admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:08.609994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Wed Jul 6 15:11:08 2022] GET /identity/v3/projects/b898c8312f89416b8b105c391af48831 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:10.568575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eaef-357e-4244-9175-cac89205fa86 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:10.569064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eaef-357e-4244-9175-cac89205fa86 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:10.569064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4eaef-357e-4244-9175-cac89205fa86 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:10.573196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:10.578391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:10.578799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:10.579045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:10.598028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:10.599589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:10.607389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 06 15:11:10.629349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4eaef-357e-4244-9175-cac89205fa86 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:10.641148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4eaef-357e-4244-9175-cac89205fa86 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:10.645940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:10.646567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:10.646852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:10.647127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:10.654702 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:10.656487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adad36-61a6-4536-8493-918308b09c9f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:10.665117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Wed Jul 6 15:11:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:11:10.670496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:10.679385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792182-b938-4c2d-ab12-d1185fa29ab4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:10.683760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:10.684615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:10.684904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:10.685181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:10.686556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:10.688406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f9903-b0f9-4ee7-93d5-0f0ea8cb9906 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:10.690454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Wed Jul 6 15:11:10 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:12.745987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2182cba-5ed8-404b-bef1-8d52788d1aeb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:12.746511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2182cba-5ed8-404b-bef1-8d52788d1aeb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:12.746605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2182cba-5ed8-404b-bef1-8d52788d1aeb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:12.747936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:12.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:12.753926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:12.754096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:12.755511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391d0f-aad6-4af7-897f-1fc90fe0c904 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:12.755799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391d0f-aad6-4af7-897f-1fc90fe0c904 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:12.756077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391d0f-aad6-4af7-897f-1fc90fe0c904 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:12.757504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:12.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:12.762426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:12.762717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:12.770714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:12.771949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:12.802630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 15:11:12.803057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 15:11:12.848309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:12.849909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:12.851349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:12.874571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:12.898579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:12.951171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:12.976175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:12.982413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c41fa-bbd0-434a-819f-60e571fd9126 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.015782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.016512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.016950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.017235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.028255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:13.039991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:13.041004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:13.041841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:13.047170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.080625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42260df6-ea16-4b5e-90aa-3b6ac174000e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.082979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.119608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.120500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.120797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.121079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.128982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.147059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225ecc-fe72-41eb-b55c-9326d080b3f8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.181302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.181977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.182298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.182585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.183403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:13.183723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.185139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.187906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.189420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.196923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:13.202146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.291695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:11:13.297453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.300710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.301869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8bff7-ceb9-4a55-be9e-3eefdccef669 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.335374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.336274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.336565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.336843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.338057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.339563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.341198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:13.346983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.361037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.401792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25235a-0d80-47fa-8f9c-bfa858815d71 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.437251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.438053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.438340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.438623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.444379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed93c5-a33d-4713-9031-4b043a1169f7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.457401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.458877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.477954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.478756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.479047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.479362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:11:13.479911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:13.480839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:13.481684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:11:13.481684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:11:13.481684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:11:13.481684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:13.481907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.481971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:13.482034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.483797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:13.486801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.489052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:13.542569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.567861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:13.586705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5cb45-7055-43de-9fbb-3564ff1014c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.587104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9c17-73bf-486a-a827-4733c7fe1732 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:13.619523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.619762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:13.620185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.620427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:13.620529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.620715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:13.620801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.620989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:13.621581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.domain.id=default, user.id=d82646fafad44bd489ae80854728f595) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:13.621708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:13.621899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.622018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:13.623362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0628a-733a-4180-babf-18e9df99586c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.623451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc675c-0ed8-421d-8c64-6f0f632d7697 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:13.630086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/role_assignments?user.id=d82646fafad44bd489ae80854728f595&scope.domain.id=default&role.id=84f17f17b9694564b251da9fa908f805 => generated 233 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:13.642216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:15.818431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb935b-3fc3-4718-8d2b-94378eeda552 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:15.818950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb935b-3fc3-4718-8d2b-94378eeda552 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:15.819045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb935b-3fc3-4718-8d2b-94378eeda552 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:15.820701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:15.826250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa0b9-a074-4863-838c-b4eab009f73e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:15.827068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa0b9-a074-4863-838c-b4eab009f73e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:15.827580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa0b9-a074-4863-838c-b4eab009f73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:15.856289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-556aa0b9-a074-4863-838c-b4eab009f73e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:15.857831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-556aa0b9-a074-4863-838c-b4eab009f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:15.935944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:15.949311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d14fd-3ff0-4b32-b6c8-c6a2fa561c10 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:15.949607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d14fd-3ff0-4b32-b6c8-c6a2fa561c10 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:15.949889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d14fd-3ff0-4b32-b6c8-c6a2fa561c10 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:15.951285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:11:15.956787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:15.957305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:15.957829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:15.984269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:15.984848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d14fd-3ff0-4b32-b6c8-c6a2fa561c10 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:15.986576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:16.050185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.117603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 12/25] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:16.158990 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.228889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d14fd-3ff0-4b32-b6c8-c6a2fa561c10 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.325804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.326615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.326903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.327185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.356272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:16.368949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:16.369792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:16.370551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 14/26] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:16.376204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.399588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.420377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492a02c-d0da-4918-8558-b3e460966eda None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.461711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.462769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.463360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.463781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.475709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:16.478191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b20e2ee2-b895-42c1-b331-b19d6193df7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:11:16.499945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:16.501489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:16.502518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:11:16.502518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:11:16.502518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:11:16.502518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:16.504864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 13/27] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:16.506645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f30c-da02-4b9e-a64c-89c5e2d5f1fa None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.507091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f30c-da02-4b9e-a64c-89c5e2d5f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.507529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66f30c-da02-4b9e-a64c-89c5e2d5f1fa None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.508925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 14/28] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:16.510544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929310f8-2914-4694-a8a7-8e6de3457213 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.510979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929310f8-2914-4694-a8a7-8e6de3457213 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.511920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929310f8-2914-4694-a8a7-8e6de3457213 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.517615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 15/29] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:16.517615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929310f8-2914-4694-a8a7-8e6de3457213 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.525567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c85b26-c08b-45d7-a459-7e79bb1d7705 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.547644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.564924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.566156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.566678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.567187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.568292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:16.568683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:16.570252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e34a48-9cd8-41ff-951d-29f5e3ff1541 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:16.581003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 15/30] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:16.582296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e061f-0f79-48e9-bd89-d18604ceb36d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.582666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e061f-0f79-48e9-bd89-d18604ceb36d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.583025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e061f-0f79-48e9-bd89-d18604ceb36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.600737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-392e061f-0f79-48e9-bd89-d18604ceb36d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:16.602062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-392e061f-0f79-48e9-bd89-d18604ceb36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:16.630378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929310f8-2914-4694-a8a7-8e6de3457213 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.668871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.669301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.670808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.671164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.671611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.672700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:16.673698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:16.675977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:16.679842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 16/31] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:16.681220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.681516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.681806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.689781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 16/32] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:16.690717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.699759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:16.700981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:16.773017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 17/33] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:16.773876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.814903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.827603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68bf75-3f1f-4bd0-b4f8-233cbe6969e3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.864041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.864922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.865216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.865502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.867183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:16.868787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:16.870535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 17/34] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:16 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:16.871295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:16.893178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e4976-9a80-42b4-aa48-aae1c081642a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:16.919985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:16.929897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:16.930673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:16.931030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:16.931399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:16.932224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:16.933829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:16.981075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec804310-ef1c-4d3a-994f-cdd993a5c5d1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:17.016412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:17.017347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.017642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.017930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.018704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:17.020186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:17.082712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 18/35] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:11:16 2022] POST /identity/v3/projects => generated 301 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:17.089952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:17.153951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 18/36] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 6 15:11:16 2022] POST /identity/v3/roles => generated 219 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:17.154943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:17.201711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:17.296675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852997ee-7ec7-4ee1-b103-496ab27a7b29 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:17.333057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c72a04-6419-404d-95dd-fb4eff8a73ee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:17.364979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:17.365691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.365984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.366271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.367132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:17.368830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:17.398443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 19/37] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 6 15:11:17 2022] POST /identity/v3/roles => generated 225 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:17.409277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:17.410098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.410486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.410778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.412347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:17.414092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39c65e-c355-4655-b976-495d3596199e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:17.416046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 19/38] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:17.423697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:17.474247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:17.563914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f809d-363b-43d2-91b8-3e11e1620fb4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.564574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f809d-363b-43d2-91b8-3e11e1620fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.565092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f809d-363b-43d2-91b8-3e11e1620fb4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.566787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 20/39] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:11:17.577155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.577782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.578338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.609285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:17.611097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:17.682050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0915c-3702-4ca3-a2f5-a7afa321f733 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:17.708066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 21/40] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 06 15:11:17.724106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:17.725033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.725324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.725608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] PATH_INFO: `/v3/domains/default/users/d82646fafad44bd489ae80854728f595/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.727747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d82646fafad44bd489ae80854728f595, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:17.729218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8799401-2c6b-4482-9748-f12d3cb80569 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:17.731071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:17.745952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 20/41] 10.222.0.46 () {58 vars in 1361 bytes} [Wed Jul 6 15:11:17 2022] PUT /identity/v3/domains/default/users/d82646fafad44bd489ae80854728f595/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:17.749901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:17.796148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd24d-7f4a-46a4-994a-fa1d36570adb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.796755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd24d-7f4a-46a4-994a-fa1d36570adb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.797181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd24d-7f4a-46a4-994a-fa1d36570adb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.798883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 21/42] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:11:17.805105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.805390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.805673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.825263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:17.826854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:17.856790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a4b60-0cdc-4e62-b9be-7dca155419c2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:17.898491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:17.900290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:17.900751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:17.901192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:17.902100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:17.903689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:17.905019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 22/43] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:17.920800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 22/44] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 6 15:11:17 2022] POST /identity/v3/roles => generated 223 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:11:17.933377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:17.966017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:18.049242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295d9a71-7339-4127-983b-f1ffa381e5bc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:18.091369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:18.092180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:18.092472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:18.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:18.093947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:18.095507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05659798-12b6-4282-b55c-98b1fc916d2f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:18.097162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 23/45] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:18.102811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:18.180847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:18.206008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205a2a1-093f-4bc1-84f5-db86dfc10a8f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:18.238746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:18.239452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:18.239746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:18.240029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:18.240760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:18.242235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57122984-4d31-4240-bded-3a57d245e676 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:18.269847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 23/46] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:11:18 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:19.767335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025939-313f-4e98-a376-ddbc64d58413 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:19.767851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025939-313f-4e98-a376-ddbc64d58413 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:19.767933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025939-313f-4e98-a376-ddbc64d58413 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:19.769280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 24/47] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:19.774648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:19.775060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:19.775319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:19.791639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:19.792845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:19.858362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 24/48] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:19.881685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca025939-313f-4e98-a376-ddbc64d58413 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:19.976125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:19.983198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca025939-313f-4e98-a376-ddbc64d58413 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.017230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.018075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.018319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.018603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.028478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0ef45123-4aa3-4718-9103-8f2b4f83c61d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:20.036187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:20.037101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:20.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 25/49] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:20.043159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.201538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:20.245675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb286e0-e154-4bf1-973d-6eef0969b996 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.278795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.279448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.279740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.280021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.280785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:20.281176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.283071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.283170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aab5-f98d-49c2-b5ab-12f9ab7069bf None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.283415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aab5-f98d-49c2-b5ab-12f9ab7069bf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.283734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175aab5-f98d-49c2-b5ab-12f9ab7069bf None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.285127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 26/50] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:11:20.289920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.290213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.290500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.295238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 25/51] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:20.301475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.308111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:20.309314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:20.376034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 27/52] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:20.397992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb114e8-05e1-4ee1-a6bf-f455aff93fa6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.398453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.415463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:20.432840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.433719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.434008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.434286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.435515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.436946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.438751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 26/53] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:20 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:20.443982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.482204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:20.497546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1ed61-cfde-4e17-830a-c82bb6c0bc33 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.530711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.531473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.531766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.532047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.533765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.535258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.537030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 28/54] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:20.541939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f09be-65a9-45e3-86f1-ca8558d7cf0d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.542127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.576242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.577071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.577395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.577688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.578876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.580896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.582679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 27/55] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:20.588068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:20.638372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb34e2-9e69-44e5-99a3-04c2e6957a13 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.641457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:20.672442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.673133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.673420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.673699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.674406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.675877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dc474-9c93-44c9-8b48-754414a13787 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.685663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94342c8-7bc4-4756-946e-bef1cf8b4cec None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:20.700275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 29/56] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Jul 6 15:11:20 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:11:20.720977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:20.721705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:20.721959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:20.722239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:20.723112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.domain.id=default, user.id=d82646fafad44bd489ae80854728f595) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:20.723463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:20.724946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283d749-2845-47bc-ae98-1b9b149b9485 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:20.735155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 28/57] 10.222.0.46 () {56 vars in 1322 bytes} [Wed Jul 6 15:11:20 2022] GET /identity/v3/role_assignments?user.id=d82646fafad44bd489ae80854728f595&scope.domain.id=default&role.id=84f17f17b9694564b251da9fa908f805 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:22.741456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368d779-1e62-4a56-907d-9a00e0938b92 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:22.741949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368d779-1e62-4a56-907d-9a00e0938b92 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:22.742032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368d779-1e62-4a56-907d-9a00e0938b92 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:22.743451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 30/58] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:22.748560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:22.748965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:22.749141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:22.765976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:22.767835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:22.832540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 29/59] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:22.856091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368d779-1e62-4a56-907d-9a00e0938b92 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:22.865297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:22.960915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368d779-1e62-4a56-907d-9a00e0938b92 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:22.993051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:22.993789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:22.994083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:22.994371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:22.995585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:22.997049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aed0a6-0fd4-4667-ac11-2d68330e64b0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:22.998657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 31/60] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:23.004704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:23.052146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:23.102075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c048107-6ee0-4af1-97e3-7e3834785b1f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:23.134947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:23.137416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:23.137710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:23.137996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:23.138709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:23.140200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86820c-650f-442c-abb8-07d193d8805f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:23.164790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 30/61] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:23 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:26.116140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195f66-6e27-47a5-9cae-db4e3c0b4852 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.117174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195f66-6e27-47a5-9cae-db4e3c0b4852 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.117693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195f66-6e27-47a5-9cae-db4e3c0b4852 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.119541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 32/62] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:26.125102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa894d-4386-4eae-8f12-95c683860647 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.126508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa894d-4386-4eae-8f12-95c683860647 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.127005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa894d-4386-4eae-8f12-95c683860647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.148595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6caa894d-4386-4eae-8f12-95c683860647 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:26.150157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6caa894d-4386-4eae-8f12-95c683860647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:26.247775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 31/63] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:26.281072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c195f66-6e27-47a5-9cae-db4e3c0b4852 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.373602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:26.407167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c195f66-6e27-47a5-9cae-db4e3c0b4852 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.442540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:26.444711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b3a-9187-4efc-b874-f0604d37740a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.445009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b3a-9187-4efc-b874-f0604d37740a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.445297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b3a-9187-4efc-b874-f0604d37740a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.446850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 32/64] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:26.447019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.447322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.447603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.451812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739101-14b3-4c07-832a-479e39c04ec4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.452103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739101-14b3-4c07-832a-479e39c04ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.452387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739101-14b3-4c07-832a-479e39c04ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.476876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9d739101-14b3-4c07-832a-479e39c04ec4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:26.478085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d739101-14b3-4c07-832a-479e39c04ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:26.480949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2d9f5647-81fc-480c-9270-13ac276df145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:26.489616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:26.490488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:26.491665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 33/65] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:26.492566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036aad4-42a0-447d-9b89-9291e07a232e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.492566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036aad4-42a0-447d-9b89-9291e07a232e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.492678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036aad4-42a0-447d-9b89-9291e07a232e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.494185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 34/66] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:26.498977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036aad4-42a0-447d-9b89-9291e07a232e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.515786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:26.567143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 33/67] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:26.569230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.569632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.570019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.596270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:26.597999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:26.623411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036aad4-42a0-447d-9b89-9291e07a232e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.659637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:26.660560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.661038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.661456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.662355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:26.662803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:26.664406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:26.666094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:26.676783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 35/68] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:26.678221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.687746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 34/69] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:26.688844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.790772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38120bf-36b1-4c0f-a734-543fd781459a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.796869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:26.804479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad08b-7351-4cb3-b239-e0e1c95c4793 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.827926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:26.829611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.830140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.830615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.841039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:26.842126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.842243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:26.842684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.843111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:26.851072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:26.852164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:26.852965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:26.853534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 36/70] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:26.854546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.863059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:26.864681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:26.866768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 35/71] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:26.867749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:26.916461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:26.961493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3067e40-69e6-4bf4-af0d-b8b837913a0a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.976714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0110e8-ded7-41d4-8703-4b157d86c0b4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:26.995197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:26.996045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:26.996335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:26.996616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.007198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:27.011480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.012157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.012450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.012742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.013502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:27.013816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.015234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e6c94-4f89-4d70-a259-8b6df67a3029 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:27.015337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:27.016133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:27.017519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:27.018154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.025551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 36/73] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.026831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76bc-cc15-4722-8a3d-57e336570e20 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.027130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76bc-cc15-4722-8a3d-57e336570e20 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.027442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76bc-cc15-4722-8a3d-57e336570e20 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.028683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 37/74] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:27.029754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc6-395c-420e-9446-dd4b5e5bbe85 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.030049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc6-395c-420e-9446-dd4b5e5bbe85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.030334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a8fc6-395c-420e-9446-dd4b5e5bbe85 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.031720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 38/75] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:27.032886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a8fc6-395c-420e-9446-dd4b5e5bbe85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.048932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.123017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f155f-03e5-4aae-b23e-826db7e3e9dc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.134080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a8fc6-395c-420e-9446-dd4b5e5bbe85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.138589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.156564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.157783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.158075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.158369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.170147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.170762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.171048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.171361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.172119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:27.172437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.173851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.181670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 39/76] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.182456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.225544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.227024 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d8b3d-67db-4919-a0b9-8919aa232fdb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.261101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.284400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5b20-9cc5-4941-8bae-21231888ec17 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.317667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.318623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.318977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.319292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.320484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.321932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f9c21-3567-4abc-ba5e-43d3db063c21 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.323571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 40/77] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.329887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.330035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 38/78] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:27.330125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.330125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.330256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.330256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.330256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.344621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:27.344905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:27.345816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:27.346098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:27.356960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.410961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 41/79] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:27.411135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 39/80] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:27.411823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.411929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.497138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.516136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1a9cf-22dd-4bf6-b77b-b9a5b108cb79 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.523387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9270ca6-69bb-4be6-922d-9c02267def0a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.549723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.550478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.550780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.551063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.561916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.562797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.563078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.563878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.565111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.566548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.566918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.572602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 42/81] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.572602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.619125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.620630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.668005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018e12a-5f32-48e3-965c-164ed39a0c30 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.701900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.702766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.703056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.703378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.704582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.706029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.707768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 43/82] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.708525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.720300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.721658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 40/83] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:27.722686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.767655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.810444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bcb82-5d51-427d-8d64-b70f59de47a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.820668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b9f8-23c4-4a26-bf31-26901d87e5d3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.843680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.844483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.844780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.845066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.853959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.854748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.855035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.855351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:27.856525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:27.856620 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:27.857991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:27.859639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 41/84] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:27.860908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:27.866174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:27.867101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:27.867917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 44/85] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:27.868540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:27.939864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.959908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0496f44-3023-4830-b62e-2c00c82797aa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.971353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3f114b-13fb-43bc-8daa-a35c85a190a0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:27.974838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:27.994470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:27.995119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:27.995432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:27.995715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.004594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.005248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.005540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.005823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.006666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:28.006989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.008482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.013778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 45/86] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.014553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.082242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.083717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.129864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5a0d1-7325-4735-b24c-02ce4445a729 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.166802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.168359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.168812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.169234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.170763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.172393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.174680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 46/87] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.175564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.178211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.202220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 42/88] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:28.203740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.220714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.288910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e435105-6f76-458e-8233-d9ed7387139c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.318206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756e7b4-9116-4c55-96d7-068e4fe3fde3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.323674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.324359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.324648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.324928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.325654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.327118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.349788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 47/89] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:28 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:28.350538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.365082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.366642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.367419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.368037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.369039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:28.369932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.372662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.385476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 43/90] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.386948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.429247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.493869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb0797-7a47-48a1-b728-ffcb87bbd779 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.504760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.536659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.537335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.537634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.537922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.538788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=d82646fafad44bd489ae80854728f595) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:28.539105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.540690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.545348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6388ce5-8e92-428a-ae9a-df1b7ed0e8d1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.546671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 48/91] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/role_assignments?user.id=d82646fafad44bd489ae80854728f595&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.547513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.587661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.588386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.588678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.588964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.590406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.592262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.595159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 44/92] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.601418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.699722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9dc7a-1a23-4a21-930c-578ce49b5289 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.737742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.742470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137629f-32c9-4f5f-90e5-50b4d5272530 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.751399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.752421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.752713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.753154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.754465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.756041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.758124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 49/93] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:28.763774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:28.782617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.783322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.783955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.784873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:28.785243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:28.786758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:28.792929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 45/94] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=7a0d559703ba47979a34ed053388473e => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:28.882720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca741b-99a3-4667-9173-ac63bc4ce068 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:28.922606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:28.923997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:28.924331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:28.924662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:28.988672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:28.999643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:29.001233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:29.105135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 50/95] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:29.116343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:29.133753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:29.266129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47af4-b378-40fb-a31f-f7046ff7b18b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:29.303273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:29.304213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:29.304507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:29.304787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] PATH_INFO: `/v3/projects/4cd43a067c3c4730afed733d2b138afe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:29.306119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] RBAC: Authorizing `identity:get_project(project_id=4cd43a067c3c4730afed733d2b138afe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:29.307487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc6033d-b745-4cce-8131-b99ed4fe3132 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:29.309199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 46/96] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:29 2022] GET /identity/v3/projects/4cd43a067c3c4730afed733d2b138afe => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:29.315242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:29.434899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:29.449480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e092a-dfa0-4522-90ad-54c16dfc96d1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:29.484986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:29.485689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:29.485981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:29.486268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:29.487119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=4cd43a067c3c4730afed733d2b138afe, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:29.487473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:29.488911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09efc-6114-4615-8eac-c4d02f13d6f3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:29.494755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 51/97] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:29 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=4cd43a067c3c4730afed733d2b138afe&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:30.956193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5f16e-0f85-48b2-8815-57b3ee8bb630 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:30.956737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5f16e-0f85-48b2-8815-57b3ee8bb630 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:30.956823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5f16e-0f85-48b2-8815-57b3ee8bb630 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:30.958299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 47/98] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:30.975966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:30.976437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:30.976534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:30.997664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:30.998876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:31.074832 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c973-acec-4c6a-81dc-0b648bc3fe9f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.075143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c973-acec-4c6a-81dc-0b648bc3fe9f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.075464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0c973-acec-4c6a-81dc-0b648bc3fe9f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.077872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 48/99] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:31.082427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.082731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.083028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.083212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 52/100] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:31.116293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:31.117506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:31.121096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.267399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:31.305264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf6a6c-92bc-46a0-8dec-627c9ceb9219 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:31.308585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 49/101] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:31.343121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.359353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:31.362582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:31.363428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.363723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.364011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.388140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:31.400556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:31.401471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:31.402318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:31.404755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 53/102] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:31.410673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.534335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfcef3-79b3-459b-9535-1a3f3242ac60 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:31.606221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:31.607028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.607343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.607628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.627130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-89a0256c-214c-46b1-ad97-64886d8359c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:31.634812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:31.635721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:31.636493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 50/103] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:31.637595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953a620-da43-412f-8b06-264a0f3f262d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.637595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953a620-da43-412f-8b06-264a0f3f262d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.637815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953a620-da43-412f-8b06-264a0f3f262d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.639059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 51/104] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:31.645346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.645636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.645917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.649861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6d1d9-373a-49f2-824e-27adefa2dc74 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:31.664183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:31.665404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:31.731610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:31.736459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:31.737483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.738001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.738550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.739626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:31.739781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 52/105] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:31.740268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:31.740595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.742163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:31.767284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 54/106] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:31.768208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.789097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:31.848967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807c8d3-3b84-4091-9515-e3a7cdd0106a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:31.886702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:31.887539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:31.887838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:31.888121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:31.888895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:31.889209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:31.890785 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:31.900992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 53/107] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:31.901852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:31.977088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:31.999902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da70d0-be9b-4c0d-953f-8075deb3af56 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.031363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f935-dc7b-4a14-9515-f13bbf0a1c68 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.047312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.049871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.050164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.050499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.065404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:32.069652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.070535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.070821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.071102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.072482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.073953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbfb34-a39d-4c66-8579-b424dbf9ede9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:32.074058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:32.074864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:32.075648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 55/108] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:32.076300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.080147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 54/109] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:31 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.081540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdac08b-4454-4d19-8a26-6d5041a671de None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.081837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdac08b-4454-4d19-8a26-6d5041a671de None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.082120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdac08b-4454-4d19-8a26-6d5041a671de None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.085638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 55/110] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:32.086374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdac08b-4454-4d19-8a26-6d5041a671de None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.156570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.195648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf8346-e103-4146-a014-c5e41bc41540 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.199719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdac08b-4454-4d19-8a26-6d5041a671de None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.226891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.233633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.234574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.234917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.235256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.236461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.237790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.238469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.238762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.239044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.239850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:32.240167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.241565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9addff-2b5f-40a2-ac97-54e23416c8c1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.243254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d23e2-e979-4850-8597-92bc3840904f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.249542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 56/111] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.249542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69fda4-fc91-41ff-873c-df616f50dbff None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.250368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69fda4-fc91-41ff-873c-df616f50dbff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.250368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69fda4-fc91-41ff-873c-df616f50dbff None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.252113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 57/112] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:32.259008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 56/113] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.259008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.259008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.259280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69fda4-fc91-41ff-873c-df616f50dbff None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.259352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.277022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:32.278332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:32.347032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 57/114] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:32.347830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.375752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69fda4-fc91-41ff-873c-df616f50dbff None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.409384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.410266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.410559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.410848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.412085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.413537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d964b25-4ac9-4932-a921-136fd7aa9a1c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.415275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 58/115] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.416423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850f7e-d05f-405f-84ce-a1bae8ca1cba None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.416721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850f7e-d05f-405f-84ce-a1bae8ca1cba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.417010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850f7e-d05f-405f-84ce-a1bae8ca1cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.424764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.436984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dc850f7e-d05f-405f-84ce-a1bae8ca1cba None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:32.438212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc850f7e-d05f-405f-84ce-a1bae8ca1cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:32.448423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c9010-51b7-4804-aca5-961edc5e90cb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.465383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.484498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.485393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.485681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.485964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.487139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.488631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.490319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 58/116] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.491076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.504975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 59/117] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:32.506300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be27d4-2153-409d-83b1-ea55e38bfe23 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.506599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be27d4-2153-409d-83b1-ea55e38bfe23 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.507124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be27d4-2153-409d-83b1-ea55e38bfe23 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.508183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 60/118] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:32.508874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be27d4-2153-409d-83b1-ea55e38bfe23 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.595076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd74e235-5e63-4033-b8b5-8baa1ab3939c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.613259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be27d4-2153-409d-83b1-ea55e38bfe23 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.629089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.629968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.630255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.630535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.631752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.633191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.634805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 59/119] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:32.639399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.642732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.648139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.649030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.649391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.649732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.659859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:32.667477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:32.668390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:32.669152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 61/120] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:32.670563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.695177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.739365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70947e49-9532-4a7c-ae7f-4db8b63d404f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.767543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fea2e-ac6d-4dff-b25e-304b432a19be None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:32.773400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.774614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.774614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.774856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/users/d82646fafad44bd489ae80854728f595/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.776922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, user_id=d82646fafad44bd489ae80854728f595, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.778372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071325f4-115a-46b9-8a55-1a31f0b35a8a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.790283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 60/121] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:32 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/users/d82646fafad44bd489ae80854728f595/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:32.791633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.791933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.792225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.810319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:32.811606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:32.815110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:32.817020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:32.817310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:32.817592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/users/a61f6086b6a94120b20b884d3e0d010e/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:32.819765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, user_id=a61f6086b6a94120b20b884d3e0d010e, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:32.821925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:32.840824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 62/122] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:32 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/users/a61f6086b6a94120b20b884d3e0d010e/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:32.841805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.904788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 61/123] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:32.905410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:32.926066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:32.965766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.007393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c6eb3-f8ef-44cf-b696-e5551d0aeee2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.036095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382e486-b714-440e-9c1e-4d01c3f038f9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.047016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.048732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.048967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.049249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.050430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.051926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.053719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 63/124] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:33.054301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.077339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.078582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.078995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.079445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.080804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.082421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.084319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 62/125] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:32 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:33.088163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.166202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b6a8-e81f-485a-b019-044fc90b7bc0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.189483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.197346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111615a-99ff-4aa7-842a-42475267f264 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.205131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.205817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.206106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.206390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.207283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:33.207606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.209096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.211819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.219043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 64/126] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:33.221190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.236933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.237939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.238328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.238709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.250588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:33.261343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:33.262199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:33.262967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:33.265460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 63/127] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:33.266521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.330993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414d3b-3201-4f70-8cb4-157eeb520046 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.375088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7f9bf-00b9-45b6-9973-e0c339c4afd6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.375290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.375839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.376133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.376420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.377407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.379785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.411728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 65/128] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:11:33 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:33.412725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.413444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.414423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.414719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.415298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/users/a61f6086b6a94120b20b884d3e0d010e/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.418945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, user_id=a61f6086b6a94120b20b884d3e0d010e, role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.420733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.445455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 64/129] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:33 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/users/a61f6086b6a94120b20b884d3e0d010e/roles/7a0d559703ba47979a34ed053388473e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:33.459356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.526503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.565928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.663311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b9aeb-c25d-43af-a445-eac3601c9875 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.681484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aceade-ea3a-44da-85ca-1fc74f9744c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.708278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.709256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.709549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.710099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.711336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:33.711788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.713601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.725841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 65/130] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:33.726437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.727444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.727807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.728177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] PATH_INFO: `/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.732804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.743940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e53db0d6448bebaa4dca6a0a47bb1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.745445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.747193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 66/131] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1 => generated 289 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:33.753457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:33.871595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.873090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:33.879170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a67bc9-55a5-4ed5-a073-979062b79d00 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.893985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4d98b-5145-4835-bcad-ff2c6b039d12 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:33.946209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.946806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.947097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.947429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.948205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:33.949249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:33.949690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:33.950211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:33.956022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:33.957799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:33.959855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 66/132] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:33.968115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:34.019489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:34.020990 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:34.147595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:34.152394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 67/133] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:34.158879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:34.271862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21538f70-6dfe-4b1c-ba94-48086e023cb9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:34.306355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa2c65-d27a-4cf5-9e68-addea46b0a26 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:34.345153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:34.346160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:34.346517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:34.346849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:34.348197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:34.349664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:34.351420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 68/134] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:34 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:34.352256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:34.353158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:34.353448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:34.353733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] PATH_INFO: `/v3/projects/4cd43a067c3c4730afed733d2b138afe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:34.356008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] RBAC: Authorizing `identity:get_project(project_id=4cd43a067c3c4730afed733d2b138afe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:34.357150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:34.357875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:34.359801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 67/135] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:33 2022] GET /identity/v3/projects/4cd43a067c3c4730afed733d2b138afe => generated 312 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:34.365790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:34.423600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:34.468725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:34.469732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de8b0-b77f-4329-9497-b700ed14a55c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:34.508312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:34.508921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:34.509282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:34.509577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:34.510494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=1f7e53db0d6448bebaa4dca6a0a47bb1) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:34.510831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:34.512415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59d023-5178-4769-a5b2-14045d67e50b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:34.518626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 69/136] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:34 2022] GET /identity/v3/role_assignments?user.id=1f7e53db0d6448bebaa4dca6a0a47bb1&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:34.635464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63471a9-10e9-4f1b-9f05-e2426992cdc1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:34.689911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:34.690834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:34.691124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:34.691432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] PATH_INFO: `/v3/projects/4cd43a067c3c4730afed733d2b138afe/users/a61f6086b6a94120b20b884d3e0d010e/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:34.696629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] RBAC: Authorizing `identity:create_grant(project_id=4cd43a067c3c4730afed733d2b138afe, user_id=a61f6086b6a94120b20b884d3e0d010e, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:34.698122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b6f68-8a8d-483b-9e20-d5eab4cdba4d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:34.725905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 68/137] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:34 2022] PUT /identity/v3/projects/4cd43a067c3c4730afed733d2b138afe/users/a61f6086b6a94120b20b884d3e0d010e/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:11:35.274346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efdd45-3203-4375-ad24-3eaf193e6bdb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.274832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efdd45-3203-4375-ad24-3eaf193e6bdb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.274911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efdd45-3203-4375-ad24-3eaf193e6bdb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.276633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 70/138] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:35.281518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.281963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.282172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.304235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:35.305512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:35.378035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 69/139] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:35.399921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efdd45-3203-4375-ad24-3eaf193e6bdb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:35.533548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efdd45-3203-4375-ad24-3eaf193e6bdb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:35.571381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:35.583637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:35.584584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.585081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.585562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.604497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:35.612901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:35.613812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:35.614611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 71/140] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:35.620100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:35.641789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:35.733376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15b2de-c35e-4400-8d28-6cd15089af63 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:35.772900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:35.773588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.773879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.774165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.774939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:35.775278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:35.776684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b253ec-fb8b-497b-aa26-fc3ef7147c92 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:35.786043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 70/141] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:35.791751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:35.898178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a62a-2ebd-44b2-a1e1-618e9f187a4e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.898500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a62a-2ebd-44b2-a1e1-618e9f187a4e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.898753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5a62a-2ebd-44b2-a1e1-618e9f187a4e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.900154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 71/142] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:35.909840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.910139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.910428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.918263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f0ec-522b-4613-ab73-98471a346c47 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:35.930058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:35.931301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:35.976068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:35.977031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:35.977315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:35.977639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] PATH_INFO: `/v3/users/d82646fafad44bd489ae80854728f595` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:35.980010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] RBAC: Authorizing `identity:get_user(user_id=d82646fafad44bd489ae80854728f595)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:35.981483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:35.983151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 72/143] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:35 2022] GET /identity/v3/users/d82646fafad44bd489ae80854728f595 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:35.988789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.009858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 72/144] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:36.063469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:36.073421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.096252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:36.168875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a93451-4457-4d78-9c8d-9a1af83f4d77 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.222709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96908000-3c59-44d4-a4c4-9733a5fc8e1d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.247973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.248658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.248948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.249228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.274329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.275144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.275599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.275893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.286688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:36.300540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:36.301387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:36.302152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 73/145] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:36.307585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.392886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:36.394294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:36.439588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:36.493872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146edaf3-15ba-4164-83d9-d6a7b27553ce None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.505838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:36.557134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.557812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.558287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.558713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.559672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:36.560126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:36.561675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:36.571388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 74/146] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:36.576899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.597300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 73/147] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:11:36.602590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.713849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab542fa-197a-4777-9d93-e661a7fa426d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.748689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.750040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.750545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.751603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.753160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:36.754970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4283-7d3d-4d81-89c7-a54d1b3daad1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:36.756942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 75/148] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:36 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:36.758346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e552a-1426-41e1-9ce1-a87d8d03ba7b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.758823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e552a-1426-41e1-9ce1-a87d8d03ba7b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.759319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e552a-1426-41e1-9ce1-a87d8d03ba7b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.760756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 76/149] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:36.768875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e552a-1426-41e1-9ce1-a87d8d03ba7b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:36.773356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:36.869380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76925b54-c451-48f2-8ca9-a8ac61811d4f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.878468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e552a-1426-41e1-9ce1-a87d8d03ba7b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:36.913555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.914770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.915308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.915779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.980013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:36.980818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.981109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.981407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:36.987963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:36.988990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:36.989426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f760c-98ab-4971-9694-12be23e02d57 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:36.990875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:36.991141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 74/150] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:36 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:36.992490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:36.992932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:36.993378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.002754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.040251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:37.041478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:37.068797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.098289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 77/151] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:37.103251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.124368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 75/152] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:37.125226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.228109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb93e17-1a99-4b40-8820-ee2bbc58f600 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.233719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c192-1d21-4086-a247-9d9337ab0b55 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.236102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.274222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.275395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.275686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.275971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.277151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.278479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.278603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.279144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.279455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.279729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.285234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 76/153] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.285354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=d82646fafad44bd489ae80854728f595) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:37.285354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.285501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.285568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6afca4-a0e2-43e4-9159-30bd27387ddc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.288965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 78/154] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/role_assignments?user.id=d82646fafad44bd489ae80854728f595&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.296191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599718-fe50-422c-966a-1056de1ff520 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.296698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599718-fe50-422c-966a-1056de1ff520 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.297134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599718-fe50-422c-966a-1056de1ff520 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.300002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 79/155] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:37.302220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c10e6-9b0b-455e-89aa-71a19b815d9d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.302670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c10e6-9b0b-455e-89aa-71a19b815d9d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.303093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c10e6-9b0b-455e-89aa-71a19b815d9d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.305358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 80/156] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:37.306586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c10e6-9b0b-455e-89aa-71a19b815d9d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.309405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.411636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca907a-8ba0-40f6-9212-7a81306b02e8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.455609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.456847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.456847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.458537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.462060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c10e6-9b0b-455e-89aa-71a19b815d9d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.470636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d7147d8f-75b9-4a42-b5d1-d70f9db51127 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:37.479740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:37.480791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:37.481691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 77/157] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:37.482947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.482947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.483268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.506170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:37.507709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:37.509780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.510450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.510735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.511015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.512153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:37.512261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.513844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692f125-89b5-4336-9bc8-2c2433475509 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.522783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 81/158] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.524390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.524687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.524974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.546825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:37.549039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:37.579305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.611415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 78/159] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:37.616612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.638076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.647039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 82/160] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:37.650059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.755271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7a36-a78f-472e-8d83-9443d4f9f5fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.785854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fb9ff-db98-4e82-bd6f-e8ea21ec047f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.793958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.794890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.795174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.795610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.796733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:37.797044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.799316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b67c8f-e0e0-4a0d-bcef-703761cbf6fb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.807437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 79/161] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.809687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0100-10a2-4152-afb7-dc80cc1be41f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.809983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0100-10a2-4152-afb7-dc80cc1be41f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.810270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c0100-10a2-4152-afb7-dc80cc1be41f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.811585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 80/162] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:37.812281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c0100-10a2-4152-afb7-dc80cc1be41f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.822777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.823674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.823965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.824244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.834180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:37.841909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:37.842770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:37.843621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 83/163] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:37.844263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:37.869570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.884933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:37.918330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c0100-10a2-4152-afb7-dc80cc1be41f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.945861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3509326-931c-4599-ba19-9978611cc2c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:37.952146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.952939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.953223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.953500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.954699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.956175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49c51e-5841-45b7-abf5-a5033bb8b9c8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.957896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 81/164] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.962426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.962426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.962426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.975588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:37.976776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:37.979435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:37.980305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:37.980590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:37.980865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:37.982047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:37.983528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:37.985694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 84/165] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:37.987421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.041307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 82/166] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:38.042379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.074328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.087478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cba07-3f24-45ab-bdf4-6fc2b082ba09 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.113900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.121270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.121945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.122239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.122524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.123320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:38.123637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.125029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.133281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 85/167] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.134057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.146254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69643a48-521c-4938-8c37-ade4e9246bb1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.180251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.180923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.181211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.181493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.182206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.183698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.203611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 83/168] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:38 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:38.204653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.237858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc34cb6-b0c9-4775-b51b-9d9559ec59de None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.285039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.286350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.286638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.287210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.313044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.322471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce022f-87c6-4f9e-93cc-c4934b483ee1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.348952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.363309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.363752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.364737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.364737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.376822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:38.384504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:38.385601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 84/169] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:38.386570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.395260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.396452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.507089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc1ee1-a0d0-443e-ba66-d2441749d9c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.528801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 86/170] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:38.532178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.556433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.557309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.557590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.558414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] PATH_INFO: `/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.559673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e53db0d6448bebaa4dca6a0a47bb1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.561195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.562964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 85/171] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.564489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.596372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.633627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.660162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad5d2d-9777-4956-9a00-e3b578e359de None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.703796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af82766-a557-47bc-8ca7-4d9caad970b1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.705003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.705811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.706193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.706798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.708662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:38.709058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.710560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.723274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 87/172] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.723274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.749945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.751718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.752226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.752720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.754150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.755848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.758188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 86/173] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.759298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.873956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53133e9-4fb1-40fd-9777-4f09fdbf38bd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.881578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d4fb4-f0da-4238-86a5-b425862f0843 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:38.903877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.914682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:38.931986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.932883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.933173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.933452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] PATH_INFO: `/v3/users/a61f6086b6a94120b20b884d3e0d010e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.935842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] RBAC: Authorizing `identity:get_user(user_id=a61f6086b6a94120b20b884d3e0d010e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.937315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.941134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 87/174] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/users/a61f6086b6a94120b20b884d3e0d010e => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.941920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:38.948190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a34e8-652f-468d-9396-18099685a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:38.949127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a34e8-652f-468d-9396-18099685a797 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:38.949414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a34e8-652f-468d-9396-18099685a797 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:38.949744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a34e8-652f-468d-9396-18099685a797 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:38.951745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a34e8-652f-468d-9396-18099685a797 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:38.953279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a34e8-652f-468d-9396-18099685a797 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:38.956654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 88/175] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:38.957390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a34e8-652f-468d-9396-18099685a797 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:39.139116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d0fd-e8c4-4c28-918a-0ab51fba3df8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:39.156719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a34e8-652f-468d-9396-18099685a797 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:39.199612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692481f6-f170-4a40-903c-63e612916abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:39.200278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692481f6-f170-4a40-903c-63e612916abb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:39.200568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692481f6-f170-4a40-903c-63e612916abb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:39.200855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692481f6-f170-4a40-903c-63e612916abb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:39.226919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:39.227785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:39.228079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:39.228362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:39.229328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:39.229751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:39.231212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:39.249146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 88/176] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=7a0d559703ba47979a34ed053388473e => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:39.249146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:39.255699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:39.279731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692481f6-f170-4a40-903c-63e612916abb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:39.281512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692481f6-f170-4a40-903c-63e612916abb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:39.355759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:39.392485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 89/177] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:39.400257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692481f6-f170-4a40-903c-63e612916abb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:39.445596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f9011-bedc-4d41-a986-78b950dfc39c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:39.529125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:39.529799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:39.535316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:39.535499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:39.535596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/users/1f7e53db0d6448bebaa4dca6a0a47bb1/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:39.540089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, user_id=1f7e53db0d6448bebaa4dca6a0a47bb1, role_id=1b9b11e67b4c4ac3bedbc1cd70fece3a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:39.541585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:39.569818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692481f6-f170-4a40-903c-63e612916abb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:39.574607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 89/178] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:39 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/users/1f7e53db0d6448bebaa4dca6a0a47bb1/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:39.636931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:39.637838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:39.638190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:39.638471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] PATH_INFO: `/v3/projects/4cd43a067c3c4730afed733d2b138afe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:39.639856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=4cd43a067c3c4730afed733d2b138afe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:39.642059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497690e4-3cb1-49b1-b95a-7c22274e3ca6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:39.644354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 90/179] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:39 2022] GET /identity/v3/projects/4cd43a067c3c4730afed733d2b138afe => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:39.650330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:39.791154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc588f2-1c2c-41af-844a-fb8af35e04a9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:39.817059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:39.835030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:39.835880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:39.836470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:39.836751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:39.837619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=4cd43a067c3c4730afed733d2b138afe, user.id=a61f6086b6a94120b20b884d3e0d010e) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:39.837942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:39.839502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca5c4bd-9cbc-45b8-b9fa-c7a8a53ae00e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:39.846681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 90/180] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:39 2022] GET /identity/v3/role_assignments?user.id=a61f6086b6a94120b20b884d3e0d010e&scope.project.id=4cd43a067c3c4730afed733d2b138afe&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:40.769636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4dee5-91b0-4d65-a1be-6a2e0e21c13e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:40.770151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4dee5-91b0-4d65-a1be-6a2e0e21c13e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:40.770226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4dee5-91b0-4d65-a1be-6a2e0e21c13e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:40.771853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 91/181] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:40.777208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701bbc-e040-407a-8da3-beb97debec37 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:40.777361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:40.777737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:40.777895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:40.778257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701bbc-e040-407a-8da3-beb97debec37 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:40.778549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc701bbc-e040-407a-8da3-beb97debec37 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:40.779920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 92/182] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:40.785753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:40.786046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:40.786571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:40.796732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:40.798463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:40.805803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:40.807283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:40.883346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 91/183] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:40.891253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 93/184] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:40.906423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:40.917044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.004196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.016741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a9d0-0213-447f-915f-25aa4051f38a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.026314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.029641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1065b4-c98f-446d-9329-42358dd83128 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.057485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.058579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.059036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.059518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.066864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.067833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.068128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.068402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.070994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5e3ecea7-6536-43bf-8e2c-c4a6c2aa8e36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:41.078863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:41.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:41.085738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:41.086336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:41.086417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 92/185] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:41.086417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f960-4fd8-40a8-98a3-3ff42349d9ee None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.086417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f960-4fd8-40a8-98a3-3ff42349d9ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.086417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f960-4fd8-40a8-98a3-3ff42349d9ee None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.086417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 93/186] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-48696841-b8d3-48a0-989d-8cbfc2944e08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:41.087617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:41.088549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:41.090916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 94/187] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:41.092890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f960-4fd8-40a8-98a3-3ff42349d9ee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.094037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.094037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.094037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.107498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:41.108760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:41.179557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 95/188] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:41.180208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.217789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f960-4fd8-40a8-98a3-3ff42349d9ee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.253165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.253792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.254078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.254359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.255122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:41.255474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.256855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.257354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.270477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 94/189] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:41.270477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.294254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b29de8-94e5-46a6-990b-683488982ef7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.302525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.331155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.331849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.332132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.332411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.333167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:41.333649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.334856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.343016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 96/190] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:41.343681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.386125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3879f-371e-454f-a3c4-91ffa14a7b4b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.420234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.421133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.421419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.421696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.423431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.424881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.430876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 95/191] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:41.430876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.461574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d01573-80ab-4795-96ae-dee8482463cd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.494217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.501490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.502381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.502669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.502950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.516764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.518232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.519974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 97/192] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:41.524479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.529912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.538244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6952cef-baa5-4b56-85c5-61cb6a074ce9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.573914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.574812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.575240 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.575529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.576708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.578180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.580025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 96/193] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:41.580606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.635427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068821f6-82bf-4758-a352-949d283afb82 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.671201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.672170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.672516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.673292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.674163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.676271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.693564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5812a3-49a5-425a-bc1a-e75605710a0e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.707088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 98/194] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:41 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:41.707835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.748840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.750536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.751010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.751506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.770995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.801192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:41.846157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.847567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13eb90-9973-48d4-9dff-03c0f52f1d48 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:41.849577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8fb4-6a2a-4753-8477-666ef90c4b24 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:41.893111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:41.893111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.893111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.893111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.967116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 97/195] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:41.968993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbc111-dfc2-46d0-88cd-0708bfb51a5b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:41.968993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbc111-dfc2-46d0-88cd-0708bfb51a5b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:41.969169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbc111-dfc2-46d0-88cd-0708bfb51a5b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:41.970989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 98/196] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:41.976758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbc111-dfc2-46d0-88cd-0708bfb51a5b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:41.982374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:41.984077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a7242-bc11-417a-ac83-49b8b90dbeff None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.041666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.056864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.106886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbc111-dfc2-46d0-88cd-0708bfb51a5b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.120362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 99/197] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:42.124522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.124813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.125094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.149979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:42.150273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.151062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.151457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.151738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.152483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:42.152922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.154360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.156284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 99/198] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:41 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:42.157572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.246822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 100/199] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:42.248098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.280314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579187d5-a9ac-4329-ae2c-adb4f8e3596b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.323096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.331679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.332851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.333342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.334338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.334739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.336871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.339136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.341281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 100/200] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:42.343509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.368409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a6e92-51b7-4e95-b2d1-4a478eff666c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.409817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.410590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.410882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.411165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.412071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=f269ffba7d254bcb807db29478f7a28a) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:42.412387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.414148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.421002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 101/201] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/role_assignments?user.id=f269ffba7d254bcb807db29478f7a28a&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:42.421805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.461719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163847c-1bcc-4bf2-8fe2-e6d790d972db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.497973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.498788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.499077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.499392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.510757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:42.519330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:42.520369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:42.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 101/202] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:42.526579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.539467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afc836-5902-4f13-8605-e9a1a9815026 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.579851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.580627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.580919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.581198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.582049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=f269ffba7d254bcb807db29478f7a28a) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:42.582365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.584008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.589836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.591321 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 102/203] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/role_assignments?user.id=f269ffba7d254bcb807db29478f7a28a&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=7a0d559703ba47979a34ed053388473e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:42.637938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed9e4c-f17b-4f5b-b3e1-7e5fbe45d54b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.728848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.729549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.729840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.730124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.730905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:42.731241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.732697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.749703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 102/204] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:42.756057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:42.760236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:42.874093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea0dd9-2ef2-4ac8-afcf-bebde193143e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:42.909278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:42.910094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:42.910378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:42.910655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] PATH_INFO: `/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:42.911885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] RBAC: Authorizing `identity:get_user(user_id=1f7e53db0d6448bebaa4dca6a0a47bb1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:42.913348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:42.915290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 103/205] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/users/1f7e53db0d6448bebaa4dca6a0a47bb1 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:42.921002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:43.131809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133ce96-05b5-4dce-968f-c455f3460ba4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:43.164171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:43.187612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:43.188330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:43.188611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:43.188887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:43.348357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:43.349849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:43.542036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 103/206] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:11:43.548498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:43.642007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:43.676401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839e4b3-aa70-4544-b65c-e9132c0cd479 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:43.728501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:43.729389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:43.729682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:43.729966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:43.731202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:43.732690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47e5b6-1dcb-4e01-84be-ba6fe5749d44 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:43.734573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 104/207] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:43 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:43.740686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:43.779885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:43.889947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703272bc-4b7f-48cf-9ebe-43f65db2098b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:43.940186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:43.940922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:43.941146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:43.941425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:43.942287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, scope.project.id=def59ab7936b4105ae0a76be3921be70, user.id=1f7e53db0d6448bebaa4dca6a0a47bb1) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:43.942605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:43.944092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eecde7d-c64b-4cc5-9c5e-d0bc181f4dfd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:43.955780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 104/208] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:43 2022] GET /identity/v3/role_assignments?user.id=1f7e53db0d6448bebaa4dca6a0a47bb1&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:44.214973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e77b-3ab1-4311-af6b-fe5578f23099 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.216013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e77b-3ab1-4311-af6b-fe5578f23099 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.216540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6e77b-3ab1-4311-af6b-fe5578f23099 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.218113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 105/209] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:44.227839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.228137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.228422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.253071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:44.254298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:44.331006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 105/210] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:44.356419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6e77b-3ab1-4311-af6b-fe5578f23099 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:44.483944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6e77b-3ab1-4311-af6b-fe5578f23099 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:44.520391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:44.521203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.521541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.521828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.528965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:44.534678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a7c05336-5640-4a76-9ae2-3f41be27c27c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:44.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:44.543640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:44.544433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 106/211] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:44.550124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:44.568635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:44.602504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7c65-93a8-40f7-8e7c-c088a30af063 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.602798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7c65-93a8-40f7-8e7c-c088a30af063 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.603082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b7c65-93a8-40f7-8e7c-c088a30af063 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.604530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 107/212] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:11:44.609550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.609883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.610174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.629366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:44.630856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:44.666062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7f346-e0e9-4087-a956-7cd5ed0a335c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:44.706486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:44.708037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.708612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.709073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.710628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:44.711407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:44.713000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:44.715096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 108/213] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:44.726393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 106/214] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:44.732072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:44.739927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:44.842202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:44.851259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267032e-60c5-43a3-a1d2-e79ce6be4cf3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:44.871567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b013-12da-4e75-91c6-8508bedfb0ba None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:44.882224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:44.892253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:44.893439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.893929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.894471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] PATH_INFO: `/v3/users/69f9d73d0af34c5e8390df4a0856f40f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.910403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] RBAC: Authorizing `identity:get_user(user_id=69f9d73d0af34c5e8390df4a0856f40f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:44.912245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:44.914530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 109/215] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:44 2022] GET /identity/v3/users/69f9d73d0af34c5e8390df4a0856f40f => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:44.919464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:44.920525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:44.920633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:44.920859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:44.921140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:44.923009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:44.924526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:44.931964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 107/216] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:44.936091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.035622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2662f-7402-4f2d-89ec-d4377e69da18 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.041691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2090338-5aff-4da7-b9b2-f2b80670e710 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.071008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.071729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.072048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.072360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.079157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.080022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.080456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.081019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.081745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.083333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa529bfd-0ca5-4e88-b3f5-7f51abdd8d38 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.105687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 108/217] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:44 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:45.106993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a807b-9d21-49ba-8647-e446573af8a0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.107314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a807b-9d21-49ba-8647-e446573af8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.107624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a807b-9d21-49ba-8647-e446573af8a0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.109030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 109/218] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:45.115049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e41d-f416-4fa8-9109-e3db84e928d1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.115409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e41d-f416-4fa8-9109-e3db84e928d1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.115676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8e41d-f416-4fa8-9109-e3db84e928d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.120899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.135244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08a8e41d-f416-4fa8-9109-e3db84e928d1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:45.136091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08a8e41d-f416-4fa8-9109-e3db84e928d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:45.156747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.173095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.174691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.212745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 110/219] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:45.220569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab83c9-a740-4aed-bf77-ed12643791bd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.220896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab83c9-a740-4aed-bf77-ed12643791bd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.221157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab83c9-a740-4aed-bf77-ed12643791bd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.223411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 111/220] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:45.232776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.233136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.233352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.256388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:45.257850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:45.291802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 110/221] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:11:45.292425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.336760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 112/222] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:45.337686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.400890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.412117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.424967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c2e9c-586d-44b7-a30d-7ded748dd9f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.463260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88ff8-c2a2-46ae-87f6-449a7de9fe4e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.472700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.473533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.473813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.474451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.488000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:45.496133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:45.497245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 111/223] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:45.498149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.503842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.505150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.505579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.506018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.507664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.509126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.510988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 113/224] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:45.511633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.613956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbca3f-223e-4d1b-b37e-762e24992297 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.646871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987840b5-1c24-4d75-9cda-b4fa24ece4ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.662397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.663209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.663536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.663814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.680338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:45.688023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:45.688938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:45.689758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 112/225] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:45.690418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.692131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.699148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.701328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.701963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.702443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.703421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:45.704436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.706049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.707109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:45.720149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 114/226] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:45.721276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.818097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b9e4-6aaa-4621-a865-ad666e1f5f31 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.830697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9d473-202c-4a72-a3a8-28f49b8d8dc3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:45.858109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.859076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.859568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.860051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.861107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=69f9d73d0af34c5e8390df4a0856f40f) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:45.861623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.863315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.869028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:45.869730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 113/227] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/role_assignments?user.id=69f9d73d0af34c5e8390df4a0856f40f&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:45.869837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:45.870886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:45.875423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:45.876476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:45.879764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:45.881423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:45.886123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:45.908195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 115/228] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:45.912568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:46.000501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:46.018137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:46.050386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73995d09-8ed4-4395-9deb-74d05e3dc73c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.068331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4a1d9-a157-47f0-bbca-2aa6156b856a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.111578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.112667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.113033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.113408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.114390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.114789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.115784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.116249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.116697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.116796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.118197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.120011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 116/229] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:46.120206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.121945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 114/230] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:45 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:46.127938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:46.129260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:46.282140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a6e31-3900-4dd2-8977-0745fdf60f6d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.287949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0b6ee-ed74-4ab8-bfaa-37e5fcda7a1f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.330254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:46.350070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.350959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.351286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.351581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.353643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.355128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.356882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 117/231] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:46 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:46.357543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.358426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.363019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.363019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.363993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.364272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:46.365474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.367349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 115/232] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:46 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:46.374086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:46.392878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:46.551469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979e975-671d-4664-8623-e14e1038fe51 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.569689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd6911-f534-4062-8af0-bbf296f4724d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:46.597558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.598463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.598755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.599038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/users/f269ffba7d254bcb807db29478f7a28a/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.606348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, user_id=f269ffba7d254bcb807db29478f7a28a, role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.607848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccecb83-6661-4d58-985f-9f668ebb3ccb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.629568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:46.634490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 116/233] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:46 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/users/f269ffba7d254bcb807db29478f7a28a/roles/7a0d559703ba47979a34ed053388473e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:46.635043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:46.635353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:46.635637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/users/f269ffba7d254bcb807db29478f7a28a/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:46.637812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, user_id=f269ffba7d254bcb807db29478f7a28a, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:46.639406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5febbb-f9c1-4a93-8656-26f81d7d1b1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:46.665575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 118/234] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:46 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/users/f269ffba7d254bcb807db29478f7a28a/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:47.471162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17300623-acd3-4ae6-94ad-73261eefd1e7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.471707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17300623-acd3-4ae6-94ad-73261eefd1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.471784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17300623-acd3-4ae6-94ad-73261eefd1e7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.473165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 117/235] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:47.478609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.478992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.479188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.501797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f094-9f55-4406-b387-037ee4de8a8f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.502019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f094-9f55-4406-b387-037ee4de8a8f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.502306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f094-9f55-4406-b387-037ee4de8a8f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.503728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 118/236] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:47.504273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:47.505487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:47.511556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.511849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.512135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.538330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:47.539589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:47.608974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:47.652982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 119/237] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:47.660031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:47.670056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 119/238] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:47.690103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:47.721534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:47.803399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6ff4-eacf-45d6-8e89-7d46f7dad3e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:47.840318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:47.841138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.841424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.841704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.846066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fc920-33c0-48c9-bbad-74e1bfec543e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:47.853020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:47.869009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:47.869904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:47.870698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 120/239] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:47.878291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:47.916474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:47.917289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:47.917580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:47.917886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:47.923944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:47.925416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:47.927110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:47.927449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 120/240] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:47.933722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.011259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aeb632-44eb-45cd-85f8-bff577dc2f61 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.066859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.081430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.083742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.084248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.085569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.086528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:48.091924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:48.093574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:48.108518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 121/241] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:48.127166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.150411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f869a7-6520-45f9-b431-b7c3ca647d35 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.224327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.225022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.225308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.225589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.226327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:48.227847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e702cf-2342-4dc3-8c72-017323a850e7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:48.283258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 121/242] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:47 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:11:48.299622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c011e-b029-4d81-ac2f-6c14cb507d73 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.301888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.343565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d433-d75a-4437-835a-e4fddc4f87cd None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.343844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d433-d75a-4437-835a-e4fddc4f87cd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.344063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926d433-d75a-4437-835a-e4fddc4f87cd None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.345986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 122/243] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:48.352587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.353073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.353362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.358571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.359498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.360503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.360792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] PATH_INFO: `/v3/users/05027add421d49f48e0943cb84f1a6bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.375635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] RBAC: Authorizing `identity:get_user(user_id=05027add421d49f48e0943cb84f1a6bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:48.377108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:48.378837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 122/244] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/users/05027add421d49f48e0943cb84f1a6bf => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:48.385390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.387148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:48.390444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:48.515626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 123/245] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:48.526664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801929f-a9c0-4504-b1f9-727588338659 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.535436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.570955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.570955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.570955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.570955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.603402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.606083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.664939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:48.666934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:48.682933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc54fa-c002-4798-9011-c5c571a79c1a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.733934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.734787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.735080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.735399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.751128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:48.759574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:48.760541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:48.761370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 124/246] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:48.769500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.797651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 123/247] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:48.803115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.887692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.918314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cd73d-e2b2-4fd8-b245-677c75453094 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.932048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:48.942865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ff270-ac17-4c64-b6c8-e350d39de175 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:48.961219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.962032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.962423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.962712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.963543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:48.963861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:48.965377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:48.975466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 125/248] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:48.980878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:48.993527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:48.994522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:48.994810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:48.995088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:48.998884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:49.000548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:49.002430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 124/249] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:49.008264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:49.092020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8f2c-52c8-4e02-a591-8428da055ef6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:49.129217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:49.130227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.130517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.130801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] PATH_INFO: `/v3/users/69f9d73d0af34c5e8390df4a0856f40f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.132167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] RBAC: Authorizing `identity:get_user(user_id=69f9d73d0af34c5e8390df4a0856f40f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:49.133736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:49.135859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 126/250] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:48 2022] GET /identity/v3/users/69f9d73d0af34c5e8390df4a0856f40f => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:49.148498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:49.173382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f704-a723-4a1f-9a34-a96bded0fb95 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:49.220067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:49.239294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:49.273659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:49.274332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.274618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.274896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.275785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=05027add421d49f48e0943cb84f1a6bf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:49.276103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:49.277548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e131928-181a-4dfa-9605-37d5ed5fdbe6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:49.291251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 125/251] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:49 2022] GET /identity/v3/role_assignments?user.id=05027add421d49f48e0943cb84f1a6bf&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:49.291251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361407-8d80-42ff-8848-fa4b1a9bb3bc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.291251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361407-8d80-42ff-8848-fa4b1a9bb3bc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.291251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361407-8d80-42ff-8848-fa4b1a9bb3bc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.291251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 126/252] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:49.308893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.309193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.309453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.339329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f0da0-d1b4-4e71-8d05-5282e93ef832 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:49.342174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:49.344094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:49.421284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:49.422099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.422460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.422762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.424931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:49.426455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:49.428647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 127/253] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:49 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:49.437750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:49.506720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 127/254] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:49.548750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:49.557090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f017ac-fc22-4400-ae8b-db240e2075c9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:49.567383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:49.606547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:49.607881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.608392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.608866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/users/69f9d73d0af34c5e8390df4a0856f40f/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.611805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, user_id=69f9d73d0af34c5e8390df4a0856f40f, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:49.613594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921942f-5bdb-406b-a337-2094ad434350 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:49.631240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 128/255] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:49 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/users/69f9d73d0af34c5e8390df4a0856f40f/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:11:49.633334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c5c6-4d6d-4c76-bd73-925d52046920 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.633825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c5c6-4d6d-4c76-bd73-925d52046920 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.634577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c5c6-4d6d-4c76-bd73-925d52046920 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.637573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 129/256] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:49.650347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.650996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.652457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.688602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:49.690332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:49.799435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3050-e07e-44f7-baab-c2986673e9de None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:49.824949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 130/257] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:49.840699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:49.853856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:49.855497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:49.856007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:49.856507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:49.867441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:49.883164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:49.896998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:49.897965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:11:49.898751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:49.904007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 128/258] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:11:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:49.910629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.098461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8fb31-4b65-4a68-a71e-372c7b358b98 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.104680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:50.114163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639f35c-a451-4710-8c13-57321ae4bc73 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.189950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.190863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.191156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.191461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.192276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:50.192595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:50.194054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:50.211267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 129/259] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:50.213804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.214867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.215375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.215950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.219236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.236756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:50.251751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:50.252693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:11:50.254054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:50.261138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 131/260] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:50.262293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.369201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:50.417027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2d97-9275-47f3-8716-f7e87e8a3a46 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.466711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1b508-fdff-47e1-929a-41c2b9f106f4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.485954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.486942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.487480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.487976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.489063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:50.489566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:50.491344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:50.510587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 132/261] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:11:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:50.517008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.540744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.541821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.542115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.542394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.545617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:50.548318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:50.550450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 130/262] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:50 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:50.553857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:50.556251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.706777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40470ceb-69a7-459d-a923-9ba80ef8c131 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.781624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.782518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.782811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.783098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] PATH_INFO: `/v3/users/f269ffba7d254bcb807db29478f7a28a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.788415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] RBAC: Authorizing `identity:get_user(user_id=f269ffba7d254bcb807db29478f7a28a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:50.789898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:50.791762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 133/263] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:50 2022] GET /identity/v3/users/f269ffba7d254bcb807db29478f7a28a => generated 283 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:50.793669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd2b10-a6b3-4fa4-b1af-06b4e023d901 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:50.798191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:50.823579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:50.883919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:50.884604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:50.884895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:50.885176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:50.947655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:50.964646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d841e2-2a72-48da-a2df-6b348782fd0c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:51.017209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:51.017888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.018176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.018456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.118362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.119907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.154717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.156245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.299864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:51.347235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 134/264] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:51.357232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:51.380013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 131/265] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 824 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:51.388844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:51.414635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:51.524207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d71e2-f237-4a27-90f2-fc2e45facc9f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:51.562361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:51.563589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.563876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.564156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.565344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.566893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8747b-da76-4a6e-be12-f3cd07375868 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.568771 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 132/266] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:51 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:51.570301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115de59-41c5-4a81-bf63-e71be3be48c9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.570734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115de59-41c5-4a81-bf63-e71be3be48c9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.571301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115de59-41c5-4a81-bf63-e71be3be48c9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.573312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 133/267] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:51.579777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ca12-7630-49cd-bf46-dd9d5eedff8a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.580077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ca12-7630-49cd-bf46-dd9d5eedff8a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.580363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ca12-7630-49cd-bf46-dd9d5eedff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.611194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a109ca12-7630-49cd-bf46-dd9d5eedff8a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:51.612823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a109ca12-7630-49cd-bf46-dd9d5eedff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:51.618135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71222df3-2d31-4206-b268-6918248b2b68 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:51.687328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:51.689364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.689846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.690300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.691861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.693586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.696656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 135/268] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:51 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:51.697879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:51.722258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 134/269] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:51.729094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4746c-107f-4ff6-9147-f5e7ba5d5d4b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.729425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4746c-107f-4ff6-9147-f5e7ba5d5d4b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.732367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4746c-107f-4ff6-9147-f5e7ba5d5d4b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.732367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 135/270] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:51.735900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4746c-107f-4ff6-9147-f5e7ba5d5d4b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:51.749044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:51.793416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:51.852666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee15cc5-818a-4e5b-a8dc-b96a08f1a652 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:51.853387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4746c-107f-4ff6-9147-f5e7ba5d5d4b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:51.895740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:51.896409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.896756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.897034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.897924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=f269ffba7d254bcb807db29478f7a28a) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:51.898420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.899756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:51.899948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38adee-5890-4129-87ac-be61cc1d7295 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.900838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.901312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.902099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.903746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=f269ffba7d254bcb807db29478f7a28a) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:51.904534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:51.906279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 136/271] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:51 2022] GET /identity/v3/role_assignments?user.id=f269ffba7d254bcb807db29478f7a28a&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=7a0d559703ba47979a34ed053388473e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:51.906755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:51.907683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:51.908053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:51.908337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:51.919128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 136/272] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:51 2022] GET /identity/v3/role_assignments?user.id=f269ffba7d254bcb807db29478f7a28a&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:51.921716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:51.946865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:51.948145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:52.090615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b098a5d-74fc-438b-838d-8c21f084b8e1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.129230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 137/273] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:52.145433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.158820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.161994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.172765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.173600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.173887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.174171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.195619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f390a073-0208-4a17-adfc-74c3d3a7be01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:52.203456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:52.204327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:52.205135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 137/274] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:52.206491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a106-58ff-49c1-b946-b91f5417cbee None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.206491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a106-58ff-49c1-b946-b91f5417cbee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.206658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3a106-58ff-49c1-b946-b91f5417cbee None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.210138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 138/275] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:52.213544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3a106-58ff-49c1-b946-b91f5417cbee None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.294927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac69f-0713-404a-89cb-3e9b0198a717 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.339091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.339934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.340218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.340498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.341982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:52.343420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc6b89-c4e5-4bed-b5ad-3df4614d2727 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:52.345090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 138/276] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:52.345433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3a106-58ff-49c1-b946-b91f5417cbee None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.346245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.346537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.346818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.368371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:52.369563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:52.386512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.387189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.387500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.387779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.388539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:52.388854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:52.390241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:52.398849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 139/277] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:52.399707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.444592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 139/278] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:52.455167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.455167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.458673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.515276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060e774-db4f-489b-a76d-b4a1bd524a20 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.544982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f498f-b2c9-4736-80a5-1d55f47a69ba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.547969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.548707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.549086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.549377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.550079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:52.551552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea85c7-69db-49f2-b37c-dc50dd835aa2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 140/279] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:11:52 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af0a1-76fd-4d8e-937a-b580039d416f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af0a1-76fd-4d8e-937a-b580039d416f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4af0a1-76fd-4d8e-937a-b580039d416f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 141/280] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:52.583805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4af0a1-76fd-4d8e-937a-b580039d416f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.587970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.588924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.589212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.589488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] PATH_INFO: `/v3/users/8272c15912a5428ea2e010e0ba87fe77` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.605667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] RBAC: Authorizing `identity:get_user(user_id=8272c15912a5428ea2e010e0ba87fe77)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:52.607147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a71fe-a658-4fab-84d3-87bd1b3c1778 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:52.614217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 140/281] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/users/8272c15912a5428ea2e010e0ba87fe77 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:52.614217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.614217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.614217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.630727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:52.631976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:52.694798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.699539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.706608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4af0a1-76fd-4d8e-937a-b580039d416f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.708645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 141/282] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:52.712102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.755184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.756037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.756325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.756601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.767971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:52.775725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:52.776602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:52.777392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 142/283] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:52.778590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.834119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cb6b7-5a09-40a6-a6c0-c9a29f693171 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.881434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.881864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.882720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.882720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.883484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, user.id=8272c15912a5428ea2e010e0ba87fe77, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:52.883802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:52.885247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:52.890627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 142/284] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/role_assignments?user.id=8272c15912a5428ea2e010e0ba87fe77&scope.system=all&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:52.891489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:52.908651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a34331-1ba2-4373-900e-14d7d0736e49 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:52.946632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:52.947471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:52.947754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:52.948028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:52.959193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:52.959723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:52.967734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:52.968692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:52.970026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 143/285] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:52.977260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.002066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.021198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db12205-e064-46a5-9c6a-0c1d3359b828 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.088758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.089442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.089729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.090009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.090941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:53.091271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.093154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.105592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 143/286] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:53.111995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.136525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945396e2-6d35-40cb-af98-823bf2cbd476 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.175348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.176126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.176480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.176766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.177534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:53.177848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.179270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.188145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 144/287] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:53.193711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.257218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477a8b8-445c-453f-abf2-2e7ebe85e76e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.287584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.323791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.324726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.326233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.326526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] PATH_INFO: `/v3/users/05027add421d49f48e0943cb84f1a6bf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.329897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] RBAC: Authorizing `identity:get_user(user_id=05027add421d49f48e0943cb84f1a6bf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.331378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.333524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 144/288] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/users/05027add421d49f48e0943cb84f1a6bf => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:53.338258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.354825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.377879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345e230-d217-4b2f-8683-d5f27d31888d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.414170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.415375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.416074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.416832 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] PATH_INFO: `/v3/users/69f9d73d0af34c5e8390df4a0856f40f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.419068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] RBAC: Authorizing `identity:get_user(user_id=69f9d73d0af34c5e8390df4a0856f40f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.421039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.427412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 145/289] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/users/69f9d73d0af34c5e8390df4a0856f40f => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:53.428860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.532326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f180d4-1ea4-4297-87a0-e36167603288 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.567093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.568183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.568676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.569152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.569616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d10b7-48d9-4c79-993e-9d58f1ded135 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.601636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.640685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.643399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.650290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.651237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.651528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.651805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.653135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.654398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.656519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 145/290] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:53.661756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.752528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 146/291] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:11:53.754141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:53.773201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.804313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:53.868815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9fa93-daf1-4d91-9041-a670f86fb157 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.906772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.907987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.908471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.908949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/users/05027add421d49f48e0943cb84f1a6bf/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.911123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, user_id=05027add421d49f48e0943cb84f1a6bf, role_id=1b9b11e67b4c4ac3bedbc1cd70fece3a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.912955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.917414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9a9a5-e697-45ac-8021-6fb633fc736f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:53.936389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 146/292] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:11:53 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/users/05027add421d49f48e0943cb84f1a6bf/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:53.968442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:53.969741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:53.970226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:53.970701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:53.972294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:53.974269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d3ded-a58e-47ae-82e0-13a90686790d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:53.976474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 147/293] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:11:53.983420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:54.118079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd6d6c-36cf-4e27-8fbc-eed686d06d69 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:54.133705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:54.159914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:54.160728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:54.161154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:54.161556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:54.162563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=69f9d73d0af34c5e8390df4a0856f40f) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:54.162984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:54.164564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d45e-e85c-4e6b-bf81-0495cb629b2c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:54.171782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 147/294] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:53 2022] GET /identity/v3/role_assignments?user.id=69f9d73d0af34c5e8390df4a0856f40f&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.030725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fc8dd-4608-43ec-a640-c2945635e87e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.031264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fc8dd-4608-43ec-a640-c2945635e87e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.031375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fc8dd-4608-43ec-a640-c2945635e87e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.033348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 148/295] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:55.038689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.039077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.039298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.058067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:55.059558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:55.081099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c092c-fa09-43b5-b6cf-bbac7b0fa0b8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.081387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c092c-fa09-43b5-b6cf-bbac7b0fa0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.081667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c092c-fa09-43b5-b6cf-bbac7b0fa0b8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.083163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 149/296] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:55.087864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.088156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.088439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.108018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:55.109373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:55.133438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 148/297] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:55.158232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.199254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 150/298] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:55.213860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.268500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6c3e-a036-49a8-80e0-e59fd7c81944 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.273723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.278119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.310962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.311723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.312005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.312277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.322586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7fdbffec-922d-4688-9bd8-8bc59cb86c48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:55.330949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:55.331870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:55.332840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 149/299] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:55.335546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a245a16-886e-4cc1-8c92-264104f72763 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.337308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65465e3d-d2d0-4e28-9b45-8c95260af3e6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.337607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65465e3d-d2d0-4e28-9b45-8c95260af3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.338718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65465e3d-d2d0-4e28-9b45-8c95260af3e6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.340215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 150/300] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:55.340840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65465e3d-d2d0-4e28-9b45-8c95260af3e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.375096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.376191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.376644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.377058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.387469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c50e5d8d-730d-4acb-b4d8-5546c150c00f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:55.395646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:55.396537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:55.397315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 151/301] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:11:55.402721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.403013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.403326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.424484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:55.426167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:55.451419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65465e3d-d2d0-4e28-9b45-8c95260af3e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.486253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.486924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.487211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.487516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.488281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:55.488583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.489951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.498330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 151/302] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.498911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.504488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 152/303] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:55.512587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.537016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.551301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.615772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71cb3-bae9-45bd-a7ed-6e2b504290dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.618102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8a2b9-6885-4c91-9a01-c7abdc77640d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.650941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.651745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.652032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.652308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.653093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:55.653420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.654521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.654724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.655430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.655718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.656018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] PATH_INFO: `/v3/users/e686063f81ab4003b94cd217dc208db7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.667255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 152/304] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.667255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.674648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] RBAC: Authorizing `identity:get_user(user_id=e686063f81ab4003b94cd217dc208db7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.676200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.682346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 153/305] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/users/e686063f81ab4003b94cd217dc208db7 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.682346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.765565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.777298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0fb16-9299-40ff-9ecf-9da703f40ed2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.779198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:55.785954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a72caa-d425-43a9-9fc0-e9f98b6b8869 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.815481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.816536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.816971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.817383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.818788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.820634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.821482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.822359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.822643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.822886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 153/306] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.823002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] PATH_INFO: `/v3/users/8272c15912a5428ea2e010e0ba87fe77` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.823907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.828104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] RBAC: Authorizing `identity:get_user(user_id=8272c15912a5428ea2e010e0ba87fe77)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.830259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.831810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 154/307] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/users/8272c15912a5428ea2e010e0ba87fe77 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.832557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:55.937523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43758290-071d-4405-8b25-12a54b9fac9f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.942973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304105d-673a-42cb-a499-5404378724f3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:55.974114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.974812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.975087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.975397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] PATH_INFO: `/v3/groups` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.976120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.977561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fede0d7-4339-4ad4-aba7-4962d3c2b320 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.977724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:55.978386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:55.978674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:55.978954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:55.979834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, user.id=e686063f81ab4003b94cd217dc208db7, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:55.980162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:55.981593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:55.988097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 154/308] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 6 15:11:55 2022] GET /identity/v3/role_assignments?user.id=e686063f81ab4003b94cd217dc208db7&scope.system=all&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:55.988878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:56.007954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 155/309] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Jul 6 15:11:55 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:56.025697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:56.124093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24dfe2-b75f-4a96-83dd-d0bca3209f16 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:56.165516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:56.166426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.166717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.167002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] PATH_INFO: `/v3/system/users/8272c15912a5428ea2e010e0ba87fe77/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.168722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8272c15912a5428ea2e010e0ba87fe77, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:56.170403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eabca-3fef-4a3c-bd27-da9bddae31a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:56.184656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 155/310] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Jul 6 15:11:55 2022] PUT /identity/v3/system/users/8272c15912a5428ea2e010e0ba87fe77/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:56.248138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45df26-596e-471c-b3cf-4f4257dea498 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.248654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45df26-596e-471c-b3cf-4f4257dea498 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.248763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45df26-596e-471c-b3cf-4f4257dea498 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.250450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 156/311] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:56.259618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.260145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.260582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.279475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:56.280713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:56.358129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 156/312] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:56.381605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45df26-596e-471c-b3cf-4f4257dea498 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:56.409402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:56.491331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45df26-596e-471c-b3cf-4f4257dea498 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:56.527992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:56.528713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.528997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.529271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.540255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:56.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:56.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:11:56.550726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 157/313] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:56.555866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:56.577019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:56.671017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d22b1c-43d2-4978-8dd6-f5051cb03b4d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:56.707039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:56.707987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.708417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.708861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.709758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:56.710212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:56.711753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:56.721312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 157/314] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:56.726901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:56.810960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:56.829459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9182b4-e64f-4b67-9817-bcfaca39dd30 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:56.863658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:56.864562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:56.864826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:56.865102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] PATH_INFO: `/v3/users/05027add421d49f48e0943cb84f1a6bf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:56.866304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] RBAC: Authorizing `identity:get_user(user_id=05027add421d49f48e0943cb84f1a6bf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:56.867793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:56.869719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 158/315] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:56 2022] GET /identity/v3/users/05027add421d49f48e0943cb84f1a6bf => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:56.874974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:56.971588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:56.980217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f838f-1977-46a6-8c86-2d1a6cd3ac09 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:57.015336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:57.016549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:57.016840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:57.017123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:57.085383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:57.086807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:57.167608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:57.191624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 158/316] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:11:57.200302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:57.310244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f0ca6-2a9c-47e5-a471-c32b8ab267b5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:57.346600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:57.347418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:57.347704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:57.348040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:57.349292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:57.350755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c9bbe-64dd-48d6-8575-2dc6c3ba520e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:57.352842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 159/317] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:11:57 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:57.358306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:57.465519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a685c6a-2add-4436-84d5-1dae145c746a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:57.501440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:57.502108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:57.502397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:57.502680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:57.503555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, scope.project.id=d89bca118a5b490f9d966eafc170457b, user.id=05027add421d49f48e0943cb84f1a6bf) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:57.503876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:57.505324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7589e-5b0c-42af-af6f-ce2ced7764d9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:57.511612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 159/318] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:11:57 2022] GET /identity/v3/role_assignments?user.id=05027add421d49f48e0943cb84f1a6bf&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:58.096751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59409-a70a-405f-87d3-66474201165a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.097176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59409-a70a-405f-87d3-66474201165a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.097352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc59409-a70a-405f-87d3-66474201165a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.098825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 160/319] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:58.103800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.104212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.104492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.121176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:58.122362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:58.159440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.192032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 160/320] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:11:58.211002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c280-92cd-45b8-893e-627a56bb2e05 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.211330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c280-92cd-45b8-893e-627a56bb2e05 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.211616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1c280-92cd-45b8-893e-627a56bb2e05 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.213157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 161/321] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:58.214314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.218866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce294-a489-497f-9742-07b7431b3185 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.219801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce294-a489-497f-9742-07b7431b3185 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.220092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bce294-a489-497f-9742-07b7431b3185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.241212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-32bce294-a489-497f-9742-07b7431b3185 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:58.241212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-32bce294-a489-497f-9742-07b7431b3185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:58.256137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.313505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 162/322] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:58.314814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2f19-71d2-496f-9b27-094b20ea8261 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.315110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2f19-71d2-496f-9b27-094b20ea8261 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.315410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e2f19-71d2-496f-9b27-094b20ea8261 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.316806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 163/323] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:11:58.321561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.321852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.322132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.324230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590bdf5-4d9d-4750-a560-a82c03a5e0f0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.338313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:11:58.339538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:11:58.361269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.361999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.362288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.362565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.372406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:58.379993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:58.380881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:58.381677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 161/324] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:58.383446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.404814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 164/325] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:11:58.405646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.414886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.483094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.484068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ee1c5-9a94-4753-889a-d67600dcbac9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.505122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc782051-fdd2-4a86-af36-8a00451ebf6f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.518713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.519535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.519814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.520092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.521235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:58.522656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:58.524276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 162/326] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:58.525019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.542300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.542970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.543288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.543567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.544313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:58.544883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:58.546244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:58.554250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 165/327] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:58.554911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.620565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.628962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b06cc-5139-44b0-8d57-938c54cfbc0d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.652537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a798-cd14-4058-8749-149a5ae01884 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.663233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.664022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.664311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.664599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.674813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:11:58.682480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:11:58.683503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:11:58.684311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 163/328] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:11:58.684976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.690942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.691660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.691938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.692220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] PATH_INFO: `/v3/groups` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.692904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:58.694335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:58.704810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.709659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 166/329] 10.222.0.46 () {60 vars in 1134 bytes} [Wed Jul 6 15:11:58 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:11:58.710727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.808960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cf63e-b7d7-49f1-a624-94936a656fd6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.826378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c2321-a9b0-4223-9bff-ea34189a792f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:58.848656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.849552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.849841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.850121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] PATH_INFO: `/v3/users/e686063f81ab4003b94cd217dc208db7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.852322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] RBAC: Authorizing `identity:get_user(user_id=e686063f81ab4003b94cd217dc208db7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:58.853933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:58.860703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 164/330] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/users/e686063f81ab4003b94cd217dc208db7 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:58.861722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.866946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:58.867667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:58.868673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:58.868673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:58.868999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:58.869316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:58.870760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:58.881354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 167/331] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:58.887620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:58.894794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.962838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:58.981262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c47f5-3c0b-4478-8df4-02505f7e7167 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:59.008728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb500b-017a-4f80-966e-b9a2525b8234 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:59.027349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:59.028329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:59.028685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:59.029027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] PATH_INFO: `/v3/system/users/e686063f81ab4003b94cd217dc208db7/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:59.031016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e686063f81ab4003b94cd217dc208db7, role_id=1b9b11e67b4c4ac3bedbc1cd70fece3a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:59.032898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:59.043092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 165/332] 10.222.0.46 () {58 vars in 1334 bytes} [Wed Jul 6 15:11:58 2022] PUT /identity/v3/system/users/e686063f81ab4003b94cd217dc208db7/roles/1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:11:59.048191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:59.049400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:59.049929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:59.050379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] PATH_INFO: `/v3/users/8272c15912a5428ea2e010e0ba87fe77` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:59.051981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] RBAC: Authorizing `identity:get_user(user_id=8272c15912a5428ea2e010e0ba87fe77)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:59.053729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6c66d-b181-4b68-97c4-1f2f7ffabb6b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:59.055772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 168/333] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:11:58 2022] GET /identity/v3/users/8272c15912a5428ea2e010e0ba87fe77 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:11:59.064378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:11:59.172943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:11:59.184764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707276-2a5c-4437-91df-d441f53bc10e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:11:59.222998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:11:59.223827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:11:59.223987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:11:59.224263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:11:59.225116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] RBAC: Adding query filter params (role.id=86d14aa8b1d24e4587187044c5bbeb72, user.id=8272c15912a5428ea2e010e0ba87fe77, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:11:59.225440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:11:59.226882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d673f-972a-48a9-819d-d004f6afd4ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:11:59.241063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 166/334] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 6 15:11:59 2022] GET /identity/v3/role_assignments?user.id=8272c15912a5428ea2e010e0ba87fe77&scope.system=all&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:01.200061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35474f-5229-4ee3-ba7b-05eb65730189 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.200575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35474f-5229-4ee3-ba7b-05eb65730189 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.200651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35474f-5229-4ee3-ba7b-05eb65730189 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.202268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 169/335] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:01.208192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271798c-d324-4763-8c58-2345bf26a028 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.208651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271798c-d324-4763-8c58-2345bf26a028 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.208651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271798c-d324-4763-8c58-2345bf26a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.236282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2271798c-d324-4763-8c58-2345bf26a028 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:01.237511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2271798c-d324-4763-8c58-2345bf26a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:01.334133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 167/336] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:01.375005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35474f-5229-4ee3-ba7b-05eb65730189 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:01.505219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35474f-5229-4ee3-ba7b-05eb65730189 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:01.537876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365172-53c8-4410-8c41-4d63ec0c655c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.538254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365172-53c8-4410-8c41-4d63ec0c655c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.538449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07365172-53c8-4410-8c41-4d63ec0c655c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.539872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 168/337] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:12:01.541324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:01.547981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533160e-f4a6-4471-8f5a-00cd7babf073 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.548308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533160e-f4a6-4471-8f5a-00cd7babf073 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.548522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533160e-f4a6-4471-8f5a-00cd7babf073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.550080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:01.551103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.551450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.551738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.568590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:01.577031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:01.577980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:01.579387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:01.584294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3533160e-f4a6-4471-8f5a-00cd7babf073 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:01.585494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3533160e-f4a6-4471-8f5a-00cd7babf073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:01.585890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 170/338] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:01.587055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:01.712234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 169/339] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:01.713908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76463-e032-4884-b165-65540337ade8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.714401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76463-e032-4884-b165-65540337ade8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.714902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a76463-e032-4884-b165-65540337ade8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.720820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 170/340] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:01.726632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.727149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.727725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.749310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:01.755566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:01.775245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:01.785370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83b9a9-2e0a-4617-9e04-aadd2514c33b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:01.830897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:01.831632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.831924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.832210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.832987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:01.833304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:01.834695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faace1fa-b3b6-4f63-a094-0d1029a73158 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:01.841363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 171/341] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:01.842552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:01.845041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 171/342] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:01.846745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383077cf-e43a-4d6d-bdbf-84c7d374ea61 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.846745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383077cf-e43a-4d6d-bdbf-84c7d374ea61 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.846945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383077cf-e43a-4d6d-bdbf-84c7d374ea61 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.851277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 172/343] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:01.853459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:01.853755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:01.854045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:01.874093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:01.876382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:01.928334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:01.966088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 173/344] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:01.967304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:01.998359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dc1fc-ac39-4f3f-bcd2-cf1baa1ca0d6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.040034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.041170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.041675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.042155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.052555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.055778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:02.064488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:02.065571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 15:12:02.066555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:02.068787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 172/345] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:02.072296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.100225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67ef4b-274a-4c97-9421-53272514a71c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.139174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.140101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.140536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.141102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.183834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23b8b9-2268-4bf7-b069-f3b8c47efc8f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.197270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.219523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.221090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.222031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.222415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.222526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.222879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.233805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:02.242356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:02.243513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:02.244617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:02.246540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 173/346] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:02.251905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.317691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.354234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 174/347] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:02.355438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.365466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed50492-a0ff-455c-8f72-f57022ce3a6a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.400477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.401447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.401835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.402213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.415129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:02.423987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:02.424988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 174/348] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:02.429713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.462363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.483548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2b7ed-bf39-47a5-baca-d2a3ffee4838 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.522566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b3248-1547-4994-87b6-df35717e537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.523707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b3248-1547-4994-87b6-df35717e537c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.524198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b3248-1547-4994-87b6-df35717e537c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.524685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b3248-1547-4994-87b6-df35717e537c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.525743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b3248-1547-4994-87b6-df35717e537c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:02.526409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b3248-1547-4994-87b6-df35717e537c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.528039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b3248-1547-4994-87b6-df35717e537c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.537600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b2846-0ab5-4e8d-91d9-855d91616cc4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.538556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 175/349] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:02.539689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b3248-1547-4994-87b6-df35717e537c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.564732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.582292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.582463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.582752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.583037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.583830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:02.584143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.585669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.594897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 175/350] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:02.595752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.657410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b3248-1547-4994-87b6-df35717e537c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.698659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.699876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.699876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.700125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.701422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.702764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.704887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 176/351] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:02.705462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.716188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc31c1-27e3-468e-b164-5b9fa0c95b63 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.727658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.766884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.768423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.769001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.769461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.770598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:02.771407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.772986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.786437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 176/352] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:02.792167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.837565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee28fc-0da2-4d9a-8c7c-5c0eabde9820 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.865113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:02.877187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.878141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.878500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.878871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] PATH_INFO: `/v3/users/e686063f81ab4003b94cd217dc208db7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:02.881395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] RBAC: Authorizing `identity:get_user(user_id=e686063f81ab4003b94cd217dc208db7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:02.883097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:02.885026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 177/353] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/users/e686063f81ab4003b94cd217dc208db7 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:02.885860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:02.938382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4035f38-2d82-46ba-9f6f-e1dbfc544492 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:02.987210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:02.987912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:02.988197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:02.988811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.010260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.034873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd7f9-8124-46d2-8b5c-d69ec396bf5f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.077982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.079721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.080003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.080298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.081695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.083166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.089835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.089835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.090944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 178/354] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:03.095270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.135063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.208919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 177/355] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:03.209767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.224486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512d5aa-0c8c-4cb1-9bcc-3cf159be6eb9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.282917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.284719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.285409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.285409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.302478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.330375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8997de-8216-4bae-88fb-902d3ed30361 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.367673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.368257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.368542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.368815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.369538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.369639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] RBAC: Adding query filter params (role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a, user.id=e686063f81ab4003b94cd217dc208db7, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:03.369946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.371031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.372267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbd59d-9231-4527-b9b1-598333139e5b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.385947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 178/356] 10.222.0.46 () {56 vars in 1308 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/role_assignments?user.id=e686063f81ab4003b94cd217dc208db7&scope.system=all&role.id=1b9b11e67b4c4ac3bedbc1cd70fece3a => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:03.387360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732afb-fde5-4f8d-a35b-72e7035bc3b6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.387657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732afb-fde5-4f8d-a35b-72e7035bc3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.387970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62732afb-fde5-4f8d-a35b-72e7035bc3b6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.389549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 179/357] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:03.390338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62732afb-fde5-4f8d-a35b-72e7035bc3b6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.393841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.512674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 179/358] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:03.513132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.537672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62732afb-fde5-4f8d-a35b-72e7035bc3b6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.573857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.583844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.584516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.584814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.585099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.585954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=def59ab7936b4105ae0a76be3921be70) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:03.586280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.588529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa753-8a5d-4353-83f5-31b9ba4e3326 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.596316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 180/359] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=7a0d559703ba47979a34ed053388473e => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:03.598414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7921-73fc-4b48-a76f-eaa619410939 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.598717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7921-73fc-4b48-a76f-eaa619410939 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.599006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a7921-73fc-4b48-a76f-eaa619410939 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.601231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 181/360] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:03.603589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfca57-3ab4-4ade-a12d-bf85d8989778 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.603894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfca57-3ab4-4ade-a12d-bf85d8989778 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.604179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfca57-3ab4-4ade-a12d-bf85d8989778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.623976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d4dfca57-3ab4-4ade-a12d-bf85d8989778 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:03.625690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d4dfca57-3ab4-4ade-a12d-bf85d8989778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:03.647295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cc2d2-9d33-4d75-896a-8aee1dacc3cb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.698571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.699521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.699808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.700087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.701676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.703748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.705829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.706145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 180/361] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:03.707266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.721074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 182/362] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:03.722646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.723623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.724232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.743130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:03.744987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:03.834591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 183/363] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:03.835524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.844714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0b6fc-68db-4629-abb5-e21c1a40efd6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:03.862957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:03.891457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:03.892274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:03.892563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:03.892844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:03.894054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:03.895551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:03.897278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 181/364] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:03.898046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:03.973298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ccbb2-e049-44f3-a45a-8af1c823a1a8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.012807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.029531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.030724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.031017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.031445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.032650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.033271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795c1a8-cb57-49c5-b511-c18ceb1e8863 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.034110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.036728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 184/365] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.036728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.081846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.082688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.082981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.083287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.097175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:04.107111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:04.108085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:04.108895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 182/366] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:04.110093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.161124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.173566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae3484-b763-4759-989b-8020172b92fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.215899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.216986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.217276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.217552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.236661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:04.244513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:04.245653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:04.246678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 185/367] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:04.247464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.256571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168943bd-9025-473f-a13e-5f6d564cb1c4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.297320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.298504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.298797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.299080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.302698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.304203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.306214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 183/368] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.307059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.314331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.373588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda251-c7b0-4fea-bbb7-08ff08984264 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.423962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.425476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.425822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.426167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.427089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=def59ab7936b4105ae0a76be3921be70) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:04.427523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.429035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.435924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 186/369] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.436867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.439691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098ee7b-1c91-4f5f-8a3f-2af2d33a6c9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.458982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.492218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.493722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.494217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.494693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.496026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:04.496818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.498748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.512668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 184/370] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.513853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.596811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cc848-c674-4704-825f-e95e1e1c0b86 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.633143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.652884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.653581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.653866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.654145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.656841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:04.656960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.658387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.659604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bf24c-4d14-4d93-9f8e-b08fdecde46e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.670280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 187/371] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.671562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.704294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.704979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.705266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.705547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.706651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=d89bca118a5b490f9d966eafc170457b) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:04.706973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.709187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:04.717095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 185/372] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:04.717981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:04.768004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:04.816640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eecab6-7775-4bc9-9dd0-74bae03d7210 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.846884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa81fe-548c-4a85-b71c-a67331d53fb1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:04.878677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.879410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.879698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.879980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.886620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:04.887663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:04.889143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:04.889723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:04.975628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:04.977455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.023604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.027614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.069313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.070784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.111013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 186/373] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:05.116631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.196341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 188/374] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:05.201341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.240392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ad9b2-c047-48ff-a789-be0f25d05b77 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.277021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.277899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.278185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.278465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] PATH_INFO: `/v3/projects/b898c8312f89416b8b105c391af48831` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.279705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] RBAC: Authorizing `identity:get_project(project_id=b898c8312f89416b8b105c391af48831)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.281205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.283261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 187/375] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/projects/b898c8312f89416b8b105c391af48831 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:05.289219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.294697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.297507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.318243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1edb2b-8641-4222-8b54-e92cba8774d6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.356171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.357107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.357391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.357668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.361173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.362696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.364783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 189/376] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:05.372491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.415435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d5d8d-eb6f-4d94-94d5-c20bd2d6c34a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.451848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.452717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.453006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.453928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] PATH_INFO: `/v3/groups/27ba41a19c074f869c7a3978d7b080ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.460267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] RBAC: Authorizing `identity:get_group(group_id=27ba41a19c074f869c7a3978d7b080ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.461751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.463800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 188/377] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/groups/27ba41a19c074f869c7a3978d7b080ba => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:05.469730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.526731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142c153-a9c7-4c6b-9251-5e84a538e1e2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.546205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.587786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef4da3-d3c2-4236-ab53-af692808b0bf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.615130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.616140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.616426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.616706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.618130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.619710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.621860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 190/378] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:05.626883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.627518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.627809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.628088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.628940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] RBAC: Adding query filter params (group.id=27ba41a19c074f869c7a3978d7b080ba, role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=b898c8312f89416b8b105c391af48831) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:05.629258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.630767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb92ff-5e45-4fe7-a490-de7b52f51811 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.632154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:05.637401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 189/379] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/role_assignments?group.id=27ba41a19c074f869c7a3978d7b080ba&scope.project.id=b898c8312f89416b8b105c391af48831&role.id=84f17f17b9694564b251da9fa908f805 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:05.663643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:05.762975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d60879-2c07-46ef-9f49-724dc72508c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:05.804799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:05.805410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:05.805697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:05.805979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:05.806847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=d89bca118a5b490f9d966eafc170457b) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:05.807169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:05.808668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bb93d-bd60-4665-934a-d5c0156bf344 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:05.816068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 191/380] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:05 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=7a0d559703ba47979a34ed053388473e => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:06.194112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95327f78-443e-4e50-9508-9a42471e8440 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:06.194473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95327f78-443e-4e50-9508-9a42471e8440 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:06.194643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95327f78-443e-4e50-9508-9a42471e8440 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:06.196154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 190/381] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:06.201489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:06.201791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:06.202079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:06.224056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:06.225264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:06.329467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 192/382] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:06.373147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95327f78-443e-4e50-9508-9a42471e8440 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:06.517957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95327f78-443e-4e50-9508-9a42471e8440 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:06.548577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:06.556275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:06.557397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:06.557881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:06.558351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:06.570252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:06.579206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:06.580113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:06.580915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:06.583549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 191/383] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:06.589592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:06.611336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:06.723953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a398ac-0aaa-43a8-bc82-8fa038441b74 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:06.764953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:06.766221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:06.766530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:06.766791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:06.767587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:06.767904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:06.769337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:06.782666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 193/384] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:06.788240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:06.910266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867ceb-6416-4faf-829b-b80b1838a6b5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:06.956542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:06.957412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:06.957705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:06.957989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:06.959172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:06.960646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:06.962268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 192/385] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:06 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:06.967951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.067828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.113982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb8465-07bc-4b42-8501-682d0f3f4e52 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:07.161917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:07.162795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.163082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.163383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.164606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:07.166064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaa3f8-b8ae-4cb0-b908-caeb0dd602e5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:07.168167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 194/386] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:06 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:07.174429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.239796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c154-3b69-427a-8c62-0f15333d5ceb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.240112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c154-3b69-427a-8c62-0f15333d5ceb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.240397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6c154-3b69-427a-8c62-0f15333d5ceb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.242193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 195/387] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:07.255269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82c0a-2289-4be1-aa0c-df95bb739416 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.255269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82c0a-2289-4be1-aa0c-df95bb739416 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.255456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.255530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb82c0a-2289-4be1-aa0c-df95bb739416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.283256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8fb82c0a-2289-4be1-aa0c-df95bb739416 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:07.283256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8fb82c0a-2289-4be1-aa0c-df95bb739416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:07.324152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6275e8-8aee-42e4-9b5f-695d60cbc2db None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:07.367907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 196/388] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:07.368114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:07.368114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.368310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452404-fbfe-44e6-9280-16607affea6b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.368393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.368393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/groups/d2bcbc808b6842228290d581a7da04e5/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.368549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452404-fbfe-44e6-9280-16607affea6b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.368549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452404-fbfe-44e6-9280-16607affea6b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.368549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 197/389] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:07.370648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, group_id=d2bcbc808b6842228290d581a7da04e5, role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:07.372295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:07.383130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.383130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.383130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.396945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 193/390] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 6 15:12:07 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/groups/d2bcbc808b6842228290d581a7da04e5/roles/7a0d559703ba47979a34ed053388473e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:07.398547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.407972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:07.413521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:07.523677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.529353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 198/391] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:07.571018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.606066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c91e4-d05f-4620-93ae-a07211fcf7c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:07.625772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.649109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:07.650169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.650649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.651107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.665782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:07.674652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:07.675685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:07.676481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:07.679327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 194/392] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:07.691270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.713245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59804a2e-6b10-4422-bdfb-167e88dffd83 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:07.786981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:07.787813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.788101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.788383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.812278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:07.820664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:07.821567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:07.822362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:07.825076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 199/393] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:07.831420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.865956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75218f87-ddcc-46d4-940d-f85040c350e3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:07.911830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.915934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:07.916670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:07.917035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:07.917256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:07.918036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:07.918379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:07.920106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:07.929841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 195/394] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:07.935293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:07.980231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:07.989058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366329-424a-4669-a920-a67ac712f0ff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.040816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.041904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.042386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.042741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179b2be-180a-480d-9039-4f4a0743eff3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.043100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.044158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:08.044927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.046527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.056127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 200/395] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:08.061758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.111831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.113081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.113590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.114057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.116500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.118190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.124327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 196/396] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:07 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:08.132119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.254911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3093d-c46b-4de4-b581-08d1183512ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.299731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:08.319412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.320416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.320800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.321195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.322503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.323115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bafcf6-9b62-4fa4-8275-7f9f32c08f60 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.324145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f70f27-f4a0-42b7-af83-ab084ae306cb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.326138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 201/397] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:08.332330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb53a-bd2b-432b-881a-c5993774c039 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.332792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb53a-bd2b-432b-881a-c5993774c039 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.333179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb53a-bd2b-432b-881a-c5993774c039 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.334553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:08.334751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 202/398] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:08.338790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb53a-bd2b-432b-881a-c5993774c039 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.371270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.373388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.373910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.374481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.376652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.379601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a04f14-8042-42b7-b79e-cb0c0d5ce518 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.381973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 197/399] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:08.384064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.384544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.384972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.408744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:08.410880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:08.466628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb53a-bd2b-432b-881a-c5993774c039 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.482219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 198/400] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:08.483257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.507716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.508628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.508919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.509264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.510460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.511994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09050a-c717-4e7b-b869-eac7219022be None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.518922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 203/401] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:08.520008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66ecea-594e-432b-93b5-a399ecc968fc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.520297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66ecea-594e-432b-93b5-a399ecc968fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.520579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66ecea-594e-432b-93b5-a399ecc968fc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.525633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 204/402] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:08.525633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66ecea-594e-432b-93b5-a399ecc968fc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.585189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:08.601739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c05dbb-a98e-46ca-a46a-8607cd403be2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.609956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:08.631438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66ecea-594e-432b-93b5-a399ecc968fc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.640815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.641706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.641997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.642277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70/groups/d2bcbc808b6842228290d581a7da04e5/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.644339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] RBAC: Authorizing `identity:create_grant(project_id=def59ab7936b4105ae0a76be3921be70, group_id=d2bcbc808b6842228290d581a7da04e5, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.645710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.667252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 199/403] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 6 15:12:08 2022] PUT /identity/v3/projects/def59ab7936b4105ae0a76be3921be70/groups/d2bcbc808b6842228290d581a7da04e5/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:08.667252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.676994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.678060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.678407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.678752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.692265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-05770068-f655-46fa-8563-ff788395f8c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:08.702214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:08.703189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:08.704092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:08.710855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 205/404] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:08.712785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.713370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.713928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.737596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:08.739292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:08.798351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57374b4b-43b0-49cd-a143-088e9a9a642b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.824645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 206/405] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:08.825777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.836607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:08.837511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:08.837800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:08.838082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/groups/d2bcbc808b6842228290d581a7da04e5/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:08.840149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, group_id=d2bcbc808b6842228290d581a7da04e5, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:08.841604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:08.858127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 200/406] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 6 15:12:08 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/groups/d2bcbc808b6842228290d581a7da04e5/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:08.879745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:08.887559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:08.956813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4f30e-906f-4990-b919-fe2172a29356 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:08.957526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:09.017738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.018416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.018698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.018976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.019784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:09.020080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.021452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.030451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 207/407] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:09.036285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.077239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f14c09-9684-4583-b568-d6663a23a156 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.117535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.118860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.119247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.119630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.133298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:09.141916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:09.143003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:09.144010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:09.145999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 201/408] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:09.152913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.173283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c624b48-d270-4365-b117-b01fd170cb51 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.193503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:09.236760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.239176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.239530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.239813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.243830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.248187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.254266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 208/409] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:09.264564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.334628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:09.361313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10237-c8f3-42b5-999b-6dcc7692d50a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.405488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.407156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.407669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.408171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.409131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:09.410067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.411714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.434769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 202/410] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:09.447544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0984c-3edd-4660-9193-eeccf0bea565 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.448804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.503576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.506105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.506663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.507147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.508568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.510450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.512752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 209/411] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:09.519086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.601083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:09.645016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b28b2-7ec8-4107-abc2-9d3f3d0b9c96 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.646322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ed504-fa9a-4481-879c-aa04c4645fae None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.710955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.711859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.712150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.712429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] PATH_INFO: `/v3/groups/27ba41a19c074f869c7a3978d7b080ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.713572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.714447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.714726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.714994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b/groups/d2bcbc808b6842228290d581a7da04e5/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.716695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] RBAC: Authorizing `identity:get_group(group_id=27ba41a19c074f869c7a3978d7b080ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.718155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.719849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 203/412] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/groups/27ba41a19c074f869c7a3978d7b080ba => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:09.721261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] RBAC: Authorizing `identity:create_grant(project_id=d89bca118a5b490f9d966eafc170457b, group_id=d2bcbc808b6842228290d581a7da04e5, role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.722679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22920397-7634-4d0b-a667-858eb687fdce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.725759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.745312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 210/413] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 6 15:12:09 2022] PUT /identity/v3/projects/d89bca118a5b490f9d966eafc170457b/groups/d2bcbc808b6842228290d581a7da04e5/roles/7a0d559703ba47979a34ed053388473e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:09.756641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:09.799303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ef5-23b8-48e7-b71e-251ddfff7a95 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.799632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ef5-23b8-48e7-b71e-251ddfff7a95 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.799865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da13ef5-23b8-48e7-b71e-251ddfff7a95 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.801643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 211/414] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:12:09.819491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.819894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.820143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.849916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:09.852036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:09.883572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e35e4-2d37-408a-af60-210d0509cc48 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:09.958451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 212/415] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:09.964168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:09.965001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:09.965293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:09.965578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] PATH_INFO: `/v3/projects/b898c8312f89416b8b105c391af48831` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:09.966893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] RBAC: Authorizing `identity:get_project(project_id=b898c8312f89416b8b105c391af48831)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:09.968378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:09.970438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 204/416] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/projects/b898c8312f89416b8b105c391af48831 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:09.975646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:09.990018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:10.011657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:10.080560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06495d24-d187-49b3-9d61-c566309c94bd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:10.115870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:10.116753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:10.117036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:10.117311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] PATH_INFO: `/v3/projects/b898c8312f89416b8b105c391af48831/groups/27ba41a19c074f869c7a3978d7b080ba/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:10.119335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] RBAC: Authorizing `identity:create_grant(project_id=b898c8312f89416b8b105c391af48831, group_id=27ba41a19c074f869c7a3978d7b080ba, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:10.120903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:10.147609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 213/417] 10.222.0.46 () {58 vars in 1442 bytes} [Wed Jul 6 15:12:09 2022] PUT /identity/v3/projects/b898c8312f89416b8b105c391af48831/groups/27ba41a19c074f869c7a3978d7b080ba/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:10.157515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:10.176972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476189-1a74-438c-ab71-afeaf590c7dc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:10.214761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:10.215651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:10.215989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:10.216271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:10.228338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:10.237209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:10.238108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:10.238926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 205/418] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:10.248387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:10.356328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:10.362615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c296c-74eb-4999-bdcb-e9d0babaf124 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:10.405760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:10.406715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:10.407168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:10.407623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:10.408536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:10.408987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:10.410501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:10.420099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 214/419] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:10.425865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:10.566945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873e8bc-b60e-4cec-b25a-e90b034fd145 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:10.620833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:10.621738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:10.622192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:10.622613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:10.643442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:10.698573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:10.700061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:10.801085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 206/420] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:10.806354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:10.957390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492419f-db75-4ed0-aa51-f9b1e718be0b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.009776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.010656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.010943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.011246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.012452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:11.013891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e8bb-5a8b-4d4a-83e2-0c2fd58b3168 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:11.016456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 215/421] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:10 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:11.021732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.063118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5925a-4e25-44be-b11e-9d418973ffc6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.063492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5925a-4e25-44be-b11e-9d418973ffc6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.063719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5925a-4e25-44be-b11e-9d418973ffc6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.065216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 216/422] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:11.070326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.070622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.070903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.100542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:11.101727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:11.103567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.146884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.201048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f081-d0f4-43b8-bc94-388039058e8c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.231404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 217/423] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:11.263040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.264281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.264522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.264997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.266082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.267955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:11.269666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:11.273166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 207/424] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:11.279631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.427943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789112e-db96-47ba-b846-8ccdb08345df None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.440910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8631a-fd20-4701-8034-2df2949533a7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.467672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.469232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.470461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.470946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.471523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.490926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:11.505866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:11.506785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:11.507621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:11.510088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 218/425] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:11.511318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.512787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.513517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.513860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.514140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.514993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=def59ab7936b4105ae0a76be3921be70) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:11.515331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:11.517340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:11.535691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 208/426] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=7a0d559703ba47979a34ed053388473e => generated 611 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:11.548237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.636670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fb98-f03a-4bd9-8cc7-81c011d8ea17 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.674756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.675483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.675771 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.676048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.676818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:11.677127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:11.678519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbc752-0977-48f0-b827-e2f597b08db1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:11.689536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 219/427] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:11.699824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.710802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b31de-c681-4430-b07d-86b9352d9fb4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.711106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b31de-c681-4430-b07d-86b9352d9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.711403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b31de-c681-4430-b07d-86b9352d9fb4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.712897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 220/428] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:11.718237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.718770 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.719288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.744857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:11.746018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:11.828381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 221/429] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:11.840993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5aaf6-5980-414b-9e99-b27649baf624 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:11.855154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:11.866942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.880723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:11.881408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:11.881692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:11.881966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:11.934958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:11.964026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:11.965494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:11.986179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d05e-eb72-4813-b53a-cc3727588337 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.022453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.023513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.023965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.024465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.036665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:12.045385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:12.046308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 15:12:12.047796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:12.050496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 222/430] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:12.056090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.086068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 209/431] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:12.091255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.148411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.192761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af39579-4519-4a2d-bb3f-f69a1810d992 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.194800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae974-917b-42db-a3d0-3f397c0180d7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.195233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.229795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.231129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.231649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.232163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] PATH_INFO: `/v3/projects/def59ab7936b4105ae0a76be3921be70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.233592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=def59ab7936b4105ae0a76be3921be70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.235908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.237254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.237939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.238238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.238540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 210/432] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/projects/def59ab7936b4105ae0a76be3921be70 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:12.242924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.242924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:12.242924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.242924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.244400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.256984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 223/433] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:12.258019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.364937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0905-8c39-4c96-b456-f5742117e6c1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.365145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59871832-401f-4f7c-be73-78d8d9bc0c70 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.394429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.400867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.401350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.401733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.402001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.402104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.402284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.402386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.402560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.403644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.405136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963141de-ec80-418d-836f-f65f5b49833c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.410637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 211/434] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:12.410637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dffda-da66-4762-85f9-7adcaba18dd5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.410637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dffda-da66-4762-85f9-7adcaba18dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.410637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dffda-da66-4762-85f9-7adcaba18dd5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.410637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 212/435] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:12.415419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dffda-da66-4762-85f9-7adcaba18dd5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.430547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.481749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.483238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608b614-8fae-4d58-8b3b-a0fdd2e040c9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.522080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dffda-da66-4762-85f9-7adcaba18dd5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.556637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.557301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.557587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.557866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.558709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=def59ab7936b4105ae0a76be3921be70) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:12.559020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.560513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7d185-b86b-4d5f-a8fa-9080cfe04b11 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.573419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 213/436] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=def59ab7936b4105ae0a76be3921be70&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:12.573419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.573419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.573419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.590907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:12.592261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:12.615168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 224/437] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:12:12.617172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df081827-4fe9-4aea-8718-1679362f9e04 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.617475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df081827-4fe9-4aea-8718-1679362f9e04 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.617762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df081827-4fe9-4aea-8718-1679362f9e04 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.619443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 225/438] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:12.625793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.626135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.626351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.646932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:12.648351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:12.683292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 214/439] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:12.683292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.683292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.688206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.742557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 226/440] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:12.743901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.800144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e92e9-7e8f-4b0b-8810-aef7d9d0fe15 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.839598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.840468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.840750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.841025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.842218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:12.843761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:12.845455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 215/441] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:12.847346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.863078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9335-f6b8-47d6-9e14-31b7db757ac9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:12.906746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:12.907533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:12.908307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:12.908307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:12.922133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:12.931004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:12.931952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 227/442] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:12.932764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:12.934828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.960119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:12.971920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62437-be71-44f5-a430-163b6a3a5fac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.010870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.011802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.012086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.012760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.027639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:13.035855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:13.036764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:13.037605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 216/443] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:13.038299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.054325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d4c9f-3047-4138-842d-652defefa775 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.098339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.099480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.099923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.100207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.101421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.102858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.104846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 228/444] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:12 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:13.105630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.151069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeb70-3a05-49e9-8674-c50c9349ffde None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.184591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.185191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.185473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.185747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.186502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:13.186808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.188964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.190313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.192749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.198839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 217/445] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:13.199846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.209042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401d064-b4d2-437d-b9b9-7dcf7ee6a10e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.244583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.245261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.245549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.245855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.246587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:13.249283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.249283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.257236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 229/446] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:13.258025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.302177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8014a2-d069-4b5a-b31b-96a836ee8f16 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.335860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.336523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.336812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.337097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.337941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=86d14aa8b1d24e4587187044c5bbeb72, scope.project.id=d89bca118a5b490f9d966eafc170457b) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:13.338253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.339735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.345439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 218/447] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=86d14aa8b1d24e4587187044c5bbeb72 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:13.359906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.360030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1422b28-a707-4586-93ac-022bcb2f795f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.395461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.396145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.396431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.396709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.417899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.427010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.472428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a90eaa-b1aa-4fee-91ce-7d5f0a933291 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.474039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.475560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.513775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.513775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.514038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.514187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.591765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 230/448] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:13.598012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.603624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.605110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.657615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.675404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.720445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 219/449] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:13.723581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a898a0c-a81e-438f-9615-86958ac43bda None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.725864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.757713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.758571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.758854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.759130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] PATH_INFO: `/v3/projects/d89bca118a5b490f9d966eafc170457b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.760383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] RBAC: Authorizing `identity:get_project(project_id=d89bca118a5b490f9d966eafc170457b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.761801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.763491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 231/450] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/projects/d89bca118a5b490f9d966eafc170457b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:13.768933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.822235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff3e498-5493-4d72-8809-4d199ee61e70 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.855850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.856756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.857262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.857596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] PATH_INFO: `/v3/projects/b898c8312f89416b8b105c391af48831` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.858787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] RBAC: Authorizing `identity:get_project(project_id=b898c8312f89416b8b105c391af48831)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.860366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.862048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 220/451] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/projects/b898c8312f89416b8b105c391af48831 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:13.865209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f36bd-567c-43df-81a6-8f093960c56f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:13.867455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.882804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.895027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:13.900157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:13.901078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:13.901375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:13.901654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] PATH_INFO: `/v3/groups/d2bcbc808b6842228290d581a7da04e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:13.902847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] RBAC: Authorizing `identity:get_group(group_id=d2bcbc808b6842228290d581a7da04e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:13.904323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:13.906008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 232/452] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/groups/d2bcbc808b6842228290d581a7da04e5 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:13.912251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:13.970362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2aa64-021e-478e-838f-82b729f68c71 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:14.003176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:14.003993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:14.004283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:14.004557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] PATH_INFO: `/v3/groups/27ba41a19c074f869c7a3978d7b080ba` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:14.005730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] RBAC: Authorizing `identity:get_group(group_id=27ba41a19c074f869c7a3978d7b080ba)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:14.007172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:14.008804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 221/453] 10.222.0.46 () {56 vars in 1180 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/groups/27ba41a19c074f869c7a3978d7b080ba => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:14.009832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a1e8a-3aba-47c2-9a13-7281cb2241a7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:14.014484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:14.043882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:14.044793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:14.045082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:14.045362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:14.046201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] RBAC: Adding query filter params (group.id=d2bcbc808b6842228290d581a7da04e5, role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=d89bca118a5b490f9d966eafc170457b) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:14.046521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:14.048014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0df71-102a-4bd5-9f4b-da6ab94e6d23 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:14.053708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 233/454] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:13 2022] GET /identity/v3/role_assignments?group.id=d2bcbc808b6842228290d581a7da04e5&scope.project.id=d89bca118a5b490f9d966eafc170457b&role.id=7a0d559703ba47979a34ed053388473e => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:14.112119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:14.118947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45ebf5-f456-4ccb-9289-e29c6cc9ac6d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:14.154614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:14.155319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:14.155611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:14.155894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:14.157327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] RBAC: Adding query filter params (group.id=27ba41a19c074f869c7a3978d7b080ba, role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=b898c8312f89416b8b105c391af48831) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:14.157647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:14.159130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4e25-848d-4d3b-a468-94439da2a4d7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:14.165731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 222/455] 10.222.0.46 () {56 vars in 1376 bytes} [Wed Jul 6 15:12:14 2022] GET /identity/v3/role_assignments?group.id=27ba41a19c074f869c7a3978d7b080ba&scope.project.id=b898c8312f89416b8b105c391af48831&role.id=84f17f17b9694564b251da9fa908f805 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:16.551576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0151b-76b1-4219-9d4e-0a099a532601 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:16.552063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0151b-76b1-4219-9d4e-0a099a532601 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:16.552063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0151b-76b1-4219-9d4e-0a099a532601 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:16.553674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 234/456] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:16.558928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640561f-aae9-475c-ae7b-c4eddf1bb3f9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:16.559817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640561f-aae9-475c-ae7b-c4eddf1bb3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:16.560247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640561f-aae9-475c-ae7b-c4eddf1bb3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:16.580345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6640561f-aae9-475c-ae7b-c4eddf1bb3f9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:16.581522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6640561f-aae9-475c-ae7b-c4eddf1bb3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:16.664565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 223/457] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:16.721234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0151b-76b1-4219-9d4e-0a099a532601 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:16.781726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96d48-c9e5-4a69-9710-6e00a2e6b452 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:16.782657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96d48-c9e5-4a69-9710-6e00a2e6b452 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:16.783139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96d48-c9e5-4a69-9710-6e00a2e6b452 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:16.785597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 224/458] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:16.791186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:16.791779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:16.792254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:16.792735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:16.816403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:16.817528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:16.863816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0151b-76b1-4219-9d4e-0a099a532601 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:16.933663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 225/459] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:16.934740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:16.935655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:16.935988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:16.936268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:16.953694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:16.955776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:16.975184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:16.983243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:16.984148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:16.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:16.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:16.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:16.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:16.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 235/460] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:16.989917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.018780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.129974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4db7e-4b27-4598-aae9-7fb00bcd6a89 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.141129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.198529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.200084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.200386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.200671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.201422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:17.201745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.203710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.207359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab49e45-ce3a-4f75-8b69-54e0f790305b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.226564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 236/461] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:17.231784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.266917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.268089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.268576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.269074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.280313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.282273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-980cbbd5-ef64-4807-a111-05b85f4556f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:17.307082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:17.308025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:17.308874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:17.308874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:17.308874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:17.308874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:17.312837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 226/462] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:17.318797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae7540-9045-4e8f-a4ae-4fb60b5ac5a5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.319460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae7540-9045-4e8f-a4ae-4fb60b5ac5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.319647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae7540-9045-4e8f-a4ae-4fb60b5ac5a5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.321136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 227/463] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:17.321698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae7540-9045-4e8f-a4ae-4fb60b5ac5a5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.392640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c794fd7-195d-4550-a3e7-1bfd4ec34626 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.421341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.435960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.436690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.436982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.437822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.438792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.440853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c95a8-85af-483a-a07a-e569110fd9fe None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.461992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae7540-9045-4e8f-a4ae-4fb60b5ac5a5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.472390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 237/464] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:17 2022] POST /identity/v3/users => generated 246 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:12:17.474170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a492-ce7e-4944-b224-f1ac502d5077 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.474507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a492-ce7e-4944-b224-f1ac502d5077 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.474784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511a492-ce7e-4944-b224-f1ac502d5077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.500265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d511a492-ce7e-4944-b224-f1ac502d5077 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:17.502191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d511a492-ce7e-4944-b224-f1ac502d5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:17.506670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.507719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.508287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.508926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.510015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:17.510606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.512606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd9466-2dee-49dd-b15d-b7e4ae074c69 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.520820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.543206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 228/465] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:17.545543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c7a65-1604-4dc1-8832-e5a17df379ba None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.546122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c7a65-1604-4dc1-8832-e5a17df379ba None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.546682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c7a65-1604-4dc1-8832-e5a17df379ba None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.548753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 229/466] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:17.549999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c7a65-1604-4dc1-8832-e5a17df379ba None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.594055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 238/467] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:17.595866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.596395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.596902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.630875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:17.632100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:17.693164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c7a65-1604-4dc1-8832-e5a17df379ba None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.714575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 239/468] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:17.715842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.732295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.739351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.740134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.740791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.741078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.742279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.743775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.770790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 230/469] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:17 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:17.770790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.819567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:17.849710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa51a8-88c6-452a-8485-f3f37339bc7b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.889387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f4655-b468-4952-9abe-eae98d54d390 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:17.902616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.903494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.904062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.904371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.920516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.922696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:17.945501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:17.946406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:17.947286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:17.947286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:17.947286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:17.947286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:17.947286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 240/470] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:17.947644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:17.948273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:17.948554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:17.948831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:17.955961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:17.966682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:17.968228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f2faf74b-816d-4f19-9a7f-77a8fa8b83eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:17.998880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:17.999893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:18.000745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:18.000745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:18.000745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:18.000745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:18.000745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 231/471] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:18.003699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d765c4-018e-49f7-ba76-eac96f8b54c0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.003994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d765c4-018e-49f7-ba76-eac96f8b54c0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.004278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d765c4-018e-49f7-ba76-eac96f8b54c0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.010939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 232/472] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:18.011968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d765c4-018e-49f7-ba76-eac96f8b54c0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.087951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:18.092262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a3f71-1bba-48be-b90c-c2f61279164a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.126622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:18.135861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.136922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.137412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.137789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.138642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:18.139055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:18.140831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf61480-9d93-462b-8fe9-c43b8b4b0050 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:18.154290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 241/473] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:18.156310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.156726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.156965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d765c4-018e-49f7-ba76-eac96f8b54c0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.157207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.175208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:18.176486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:18.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.201628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.202058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.202727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.204047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:18.204780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:18.206967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:18.222904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 233/474] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:18.224416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.257614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 242/475] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:18.258668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.368827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55442e4b-5aff-4e14-8eda-a1960263b807 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.369588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:18.375097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6257-3f31-426f-9cd9-94aa7f4cf315 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.393017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:18.409220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.410029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.410415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.410807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.411654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:18.413204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:18.423254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.424246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.424670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.425810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.426867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:18.428888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:18.439147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 243/476] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:18 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:18.440493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.464473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 234/477] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:18 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:18.597392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c681-d347-4d41-b95b-91aae18074e8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.635547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.636304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.636598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.636882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.637656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:18.639077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:18.649908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 15:12:18.649908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 15:12:18.649908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [parameters: {'id': '5f7f42aae34a4595a0e9a9371e21ba8f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 15:12:18.649908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:12:18.668025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return ref.to_dict() Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:12:18.668930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 15:12:18.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 15:12:18.672333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:12:18.673563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:12:18.675707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': '5f7f42aae34a4595a0e9a9371e21ba8f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:18.676515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 15:12:18.677362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:18.678157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 244/478] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 6 15:12:18 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:18.678219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.796465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e09c63-1463-492e-b8e1-c3ae95370aa0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:18.835196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:18.836050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:18.836341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:18.836623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:18.848001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-791050bc-1f2c-4b16-bead-d6c0aaade2ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:18.856419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:18.857316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:18.858118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 235/479] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:18.864865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:18.921196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:18.999264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fcce-ad3c-445d-9326-36e6b67ca097 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:19.047985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:19.048584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:19.048872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:19.049151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:19.049911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:19.050222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:19.051615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64671c-995f-4d08-8506-0d57134fbdca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:19.060265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 245/480] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:19.844937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe8dc5-1284-4e30-85ff-720f65e895e7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:19.845481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe8dc5-1284-4e30-85ff-720f65e895e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:19.845481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe8dc5-1284-4e30-85ff-720f65e895e7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:19.847087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 236/481] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:19.852402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:19.852799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:19.852981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:19.880248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:19.881450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:19.969522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 246/482] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:20.006872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe8dc5-1284-4e30-85ff-720f65e895e7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.036909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:20.147043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe8dc5-1284-4e30-85ff-720f65e895e7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:20.188451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:20.189586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.190070 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.190568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.207470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0848ac94-fe4e-480e-aace-e7eda1a8ec08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:20.215527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:20.216422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:20.217216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:20.220057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 237/483] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:20.225922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.276314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:20.347041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d69d2-2ccd-41ee-9f7c-cf7004bd12e6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.347373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d69d2-2ccd-41ee-9f7c-cf7004bd12e6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.347727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d69d2-2ccd-41ee-9f7c-cf7004bd12e6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.349294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 238/484] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:20.353793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.354085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.354372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.365790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a4eea-ad9b-438f-bc47-2d0992c95505 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:20.384610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:20.385817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:20.424042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:20.424646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.424934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.425214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.425978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:20.426292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:20.427697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:20.437331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 247/485] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:20.442458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.517113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 239/486] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:20.540019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.559559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:20.654097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd11c47-fab7-4c8a-96de-20f92f35b5d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:20.683675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:20.691526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:20.692341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.692626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.692903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.703792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:20.711710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:20.712625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:20.713624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:20.714712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ce6c7-9f4b-4790-967f-9e5131416305 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:20.714795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 240/487] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:20.718876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.786936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:20.787774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:20.788104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:20.788369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:20.812898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:20.814362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:20.851583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:20.855084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:20.856202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:20.857072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:20.857072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:20.857072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:20.857072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:20.863577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 248/488] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:20.868851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:20.932760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4ff07-781b-46bc-8746-e954920284e7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.010326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0b7e9-3078-4c48-9aa5-0a80e0f44901 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.011179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.012119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.012620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.013093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.014068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:21.014574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.016253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b203c2-820f-4687-ac7f-9f9583963f14 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.025565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 241/489] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:21.027279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c2e8-f328-49b3-82fc-79659e98a60f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.027794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c2e8-f328-49b3-82fc-79659e98a60f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.028272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c2e8-f328-49b3-82fc-79659e98a60f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.034509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 242/490] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:21.035532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c2e8-f328-49b3-82fc-79659e98a60f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.048483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.049168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.049440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.049718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.050747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:21.051070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.052739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377692f1-fb62-4b1a-9c44-d05334630401 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.067796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 249/491] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:21.069316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.069605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.069884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.088751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:21.090544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:21.110214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.148991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c2e8-f328-49b3-82fc-79659e98a60f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.173229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 250/492] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:21.174347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.183958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.191482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.192278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.192563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.192839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.205023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.206485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e3cbd82e-560d-4c76-baf3-7370189978a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:21.224233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:21.225089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:21.225942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:21.225942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:21.225942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:21.225942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:21.225942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 243/493] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:21.229898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25b8a-8f90-4ee1-bce8-5b1860379d81 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.229898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25b8a-8f90-4ee1-bce8-5b1860379d81 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.229898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe25b8a-8f90-4ee1-bce8-5b1860379d81 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.231118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 244/494] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:21.231777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25b8a-8f90-4ee1-bce8-5b1860379d81 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.308559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221e350-301b-40be-a4e2-ed09502ffcff None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.346142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.347233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.347659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.348100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.349977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe25b8a-8f90-4ee1-bce8-5b1860379d81 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.351727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.357756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.359696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:21.377019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:21.378125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:21.379254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:21.381250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 251/495] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:21.382160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.388258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.389065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.389360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.389640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.401269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5cb93c14-0fbd-4ee6-bc54-d670377d016f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:21.409613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:21.410503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:21.411340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 245/496] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:21.412024 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.412413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.412590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.429318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:21.430501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:21.465100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.497976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 246/497] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:21.498823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.499493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca659-4b33-4f0b-8b1b-581a7b9e3332 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.547746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.548790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.549370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.549718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.550449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:21.550760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.552283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.568682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 252/498] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:21.568682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.614682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96ba42-c917-477b-925d-e233e1d943ac None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.626481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.651936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.652613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.652899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.653175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.654025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:21.654340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.656346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.673623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19833-cf63-4428-8d22-ae044af7fa74 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.693960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 247/499] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:21.694809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.712656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.713434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.713718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.713993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.714754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:21.715062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.716587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.722023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.725668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 253/500] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:21.726336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.797122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1e28e-5920-4b39-88ff-ab9825f491fa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.825235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133ea1-5aea-43d1-b5e9-06613710f3f6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.830703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.831533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.831821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.832101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.841985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.842198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:21.852917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:21.854551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 248/501] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:21.855486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.862030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.862835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.863123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.863439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.870953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:21.872422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-48426f44-4f6e-4400-a179-332a3faef982 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:21.886841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:21.888008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 254/502] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9ba0-2770-43ff-a7b6-b1738cfe839d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.894281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9ba0-2770-43ff-a7b6-b1738cfe839d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.895124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9ba0-2770-43ff-a7b6-b1738cfe839d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:21.895124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 255/503] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:21.895124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f9ba0-2770-43ff-a7b6-b1738cfe839d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:21.949314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:21.962362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f241d1d-69fe-442d-bb1c-157492e39506 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.992975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f9ba0-2770-43ff-a7b6-b1738cfe839d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:21.995357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:21.996019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:21.996305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:21.996585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.025863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.026652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.026931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.027206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.037447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.038945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:22.055598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:22.056498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:22.057339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:22.057339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:22.057339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.057339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:22.057339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 256/504] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:22.057614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.067958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.069433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.090205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.153376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.159873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7f2aa-2eee-4178-adba-6ef3fcc25bc9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.166257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 249/505] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:22.167066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.194000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.194680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.194970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.195281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.196040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:22.196362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.197726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e68bd-23f0-441d-a03a-e2499f63010c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.205493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 257/506] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:22.206660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.206954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.207259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.222712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:22.223941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:22.269715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472d966-43eb-466e-8cdb-d7d36ce48900 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.290064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 258/507] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:22.290879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.305436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.306116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.306400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.306679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.307556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:22.307866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.309307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.316532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.327453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 250/508] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:22.328455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.373117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.391543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59d82a-fd26-475f-8010-216e593d9edc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.424569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.425228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.425529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.425787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.425950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff48505-d91b-447b-a1fb-1e9259d29364 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.426513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:22.426821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.428217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.439850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 259/509] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:22.440630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.460047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.460842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.461130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.461407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.472463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.473924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:22.491327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:22.492235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:22.493083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:22.493083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:22.493083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.493083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:22.493083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 251/510] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:22.493360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.521815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.539938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671ddb5-f5e0-49d9-90d5-ccaedf5d18fd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.572880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.573665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.573951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.574240 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.584477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.585917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.587014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.591397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749186b8-eb24-4d1e-bd8d-c6955e14f74d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:22.603186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:22.604207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:22.605065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:22.605065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:22.605065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.605065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:22.605065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 260/511] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:22.605350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.628906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.629634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.629919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.630192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.640392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.641822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:22.658900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:22.659815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:22.660649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:22.660649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:22.660649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:22.660649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:22.660649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 252/512] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:22.660946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.707398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016dca-7bcd-4593-99f9-41d10f9d07ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.740670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.741317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.741605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.741885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.744519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.762679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd727b5-7205-412a-a447-2695e286a78b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.795993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.797325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.797612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.797892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.805489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.806942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.808587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.810031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:22.822198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:22.823090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:22.824050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 253/513] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:22.824747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.836970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.914922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 261/514] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:22.915885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:22.924635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80658-0b1f-43eb-8c4e-7cefea9cc375 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:22.948903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:22.958088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:22.958737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:22.959019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:22.959330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:22.960065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:22.960376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:22.961767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:22.973970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 254/515] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:22.974745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.018264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28461657-5e5a-4b72-a2b8-3e11161e090d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.043393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.051182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.051873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.052156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.052456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.053200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.053576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.054938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.066072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 262/516] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.066833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.079756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3741-febc-4dcf-b1cb-6ca8f76f39d3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.112097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.112757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.113044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.113323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.114049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.114360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.115775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.127344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 255/517] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.128024 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.162153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.169672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2c99d-66e0-4b0b-8b4f-b43a67fac7ad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.201480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.202139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.202422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.203298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.204139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.204453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.205871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.224959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc07e12-67cc-4caa-8368-36d14eb8a56f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.241088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 263/518] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.241656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.257992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.264064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.264859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.265735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.266017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.275856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.277303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:23.294511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:23.295547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:23.296461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:23.296461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:23.296461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:23.296461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:23.296461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 256/519] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:23.296720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.341827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ba269-f917-49e9-a216-07c7654ffd48 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.375251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.376250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.376528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.376805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.383276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.384712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.394323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45053-d5e6-459a-9a90-a4d92e7c5f0b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:23.398996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:23.399855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:23.400691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 264/520] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:23.401409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.419581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.429422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.430167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.430449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.430727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.438577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.440036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:23.454297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:23.455198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:23.456136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 257/521] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:23.456703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.496897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.506707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8e430-dc00-4e09-9c45-effe78ba0b5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.540245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.540909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.541194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.541468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.542169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.543633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.553706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1efd18-f8c0-4170-baa9-f1a6a3661d12 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.563412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 265/522] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:23 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:23.564358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.592216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.593354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.593842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.594266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.649046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.675234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.676755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.694554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fed5e-8bb9-49f0-9ad8-4dc497d43129 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.725538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.735611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.736290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.736577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.736857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.737598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.737910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.739350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.752086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 266/523] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.753776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.793192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 258/524] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:23.794042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.864244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f96a5-c621-416d-9056-d5da599db62a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.890441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.897078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f040d-f70e-4ed0-8fa6-bdbd4f339115 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:23.898762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.899483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.899759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.900039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.900835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.901155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.902594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.920585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 267/525] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.922258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:23.943507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:23.944187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:23.944476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:23.944808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:23.945638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:23.945930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:23.947395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:23.961482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:23.970714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 259/526] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:23.971939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.042713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b5e63-e4a7-45f4-ba0d-828daadcdbaf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.079437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.080233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.080518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.080793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.081617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a179db-da7b-438e-9cd2-1a24298778f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.088509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.090086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:24.105307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:24.106190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:24.107031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 268/527] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:24.107978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.123109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.124038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.124328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.124647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.135133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.136622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:24.154482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:24.155393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:24.156880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:24.156880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:24.156880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:24.156880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:24.156880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 260/528] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:24.157171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.164558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.212647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.219733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce6d8c-8762-4722-85ce-9906e71837ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.259551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.260216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.260505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.260785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.261624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:24.261941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.263417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.268790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694737eb-ce61-428f-90de-a4b0decebaed None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.271741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 269/529] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:24.272216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.316803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.318174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.319624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.320114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.393276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205549-2804-442e-92e5-1f10fe90c48c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.408022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.423640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.425110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.440941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.442412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.443129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.443640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.444870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:24.445758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.447425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.465650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 270/530] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:24.466701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.487411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.540768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 261/531] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:24.541710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.605721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb26029-f8af-4832-ba3c-301f195cfda2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.650646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567e6c7-c4c1-46a4-8ff4-702446a0734f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.651730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.653714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.654211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.654711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.655741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:24.656757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.658331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.675093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 271/532] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:24.676949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.688644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.689305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.689588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.689861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.690876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=f11ea126550c4c4a8892522e01ed149a) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:24.691192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.693020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.703640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 262/533] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/role_assignments?user.id=f11ea126550c4c4a8892522e01ed149a&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:24.704714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.720244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.750714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:24.871071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09267d-c27e-49bb-9fcb-c01d0cd38e25 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.899046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b855da-48cc-4f49-b5bb-bd56fb47f71c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:24.909162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.910257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.910764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.912162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.925300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.928330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.940368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:24.941202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:24.941519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:24.941798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:24.948979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:24.950432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:24.951409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:24.952214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:24.953642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:24.953642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:24.953642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:24.953642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:24.955903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 272/534] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:24.964199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:24.966605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:24.967430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:24.968318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 263/535] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:24.974323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:25.117260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:25.128014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:25.157698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027d1a2-d47e-4687-aea6-ba949fd85cef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.169651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0900-acad-49a3-a6ac-f3f3dd1ef201 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.208340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.209042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.209327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.209609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.210410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:25.210725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.212345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.237764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 264/536] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:25.243466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:25.267241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.268404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.268978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.269249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.269983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:25.270294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.271754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.289318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 273/537] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:25.295203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:25.440088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc69-d3b3-404d-bd6d-2c6f39982d7a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.463488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeb259-5083-4fd7-bf99-f3c91e6cc0ae None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.480481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.481334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.481625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.482015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.504600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.506158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.520492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.521237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.521527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.521808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.522684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=1c6dce92d6724fbf878faba138471bb8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:25.523010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.524517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.527899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:25.530446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:25.531609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:25.532479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:25.533367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 274/538] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:25.539975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 265/539] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:25 2022] GET /identity/v3/role_assignments?user.id=1c6dce92d6724fbf878faba138471bb8&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:25.540475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:25.554805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:25.682199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1964627-8fea-4c09-a039-c977ee03ca7b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.725888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.726983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.727505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.727988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.729039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:25.729584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.732046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea4298-80da-434d-b376-b42239929a95 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.752874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 275/540] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:25.758733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:25.820950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a39ed8-ad57-48a8-a62c-850c4caf80ec None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.821658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a39ed8-ad57-48a8-a62c-850c4caf80ec None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.822160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a39ed8-ad57-48a8-a62c-850c4caf80ec None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.824003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 276/541] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:25.829352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.829883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.830443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.857064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:25.858587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:25.884793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4143f8-b545-426e-bcb3-23cb1e7b76aa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:25.918851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:25.919769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:25.920060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:25.920343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:25.923256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=edbb85bda10d45bf8f66c6ee0561c2a2) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:25.923256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:25.923636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:25.929883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 266/542] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:25 2022] GET /identity/v3/role_assignments?user.id=edbb85bda10d45bf8f66c6ee0561c2a2&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:25.995618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:26.014200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 277/543] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:26.054763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:26.141385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:26.185416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de54a9d-feaf-45d0-bc52-89a1a070a02a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:26.222800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:26.223875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.224306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.224730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.236406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:26.244465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:26.245350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:26.246153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:26.248399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 267/544] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:26.253916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:26.354934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17babadd-7ef7-47c3-a3cc-8f7f7f2b1809 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:26.390077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:26.391003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.391490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.391931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.392851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:26.393312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:26.394828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:26.403717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 278/545] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:26.412471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:26.539348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae36df6-d17b-4dff-be60-bd0986d4385f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:26.564117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:26.576616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:26.577716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.578153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.578588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.590096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:26.591776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ec946043-da22-41e7-b6d8-55ee5363b0be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:26.610317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:26.611362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:26.612421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:26.612421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:26.612421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:26.612421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:26.613959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 268/546] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:26.619508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:26.714695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:26.777885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e7471-ddc2-4d05-b218-ce39ddb8b40b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:26.825897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404bce-6dcd-4f23-b87f-fd0461ef68d6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.826216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404bce-6dcd-4f23-b87f-fd0461ef68d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.826472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404bce-6dcd-4f23-b87f-fd0461ef68d6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.827984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:26.828213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 269/547] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:26.829065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.829541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.830026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.830957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:26.831505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:26.833093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:26.840401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:26.840722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:26.840976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:26.855365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 279/548] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:26.861550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:26.878914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:26.880507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:27.005333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 270/549] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:27.039479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87924e36-ade9-4fd5-8c05-7020dbc2094e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.054035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.083573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.114666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.115422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.115706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.115983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.130642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:27.132579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:27.159090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:27.160649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:27.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:27.168689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 280/550] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:27.173906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.252596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b23c4-f840-4db4-a5ba-e1e6a0b8aa39 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.264026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.332676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cde31c-0ffe-427b-ade1-45e4f84d7296 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.337001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.337946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.338334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.338720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.350369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1cf6f2db-38ff-4450-ab12-d16c00c4bf28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:27.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:27.359958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:27.360961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:27.363545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 271/551] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:27.364934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68846-369d-43a8-ab30-d4fdcdb2b75b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.365273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68846-369d-43a8-ab30-d4fdcdb2b75b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.365654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d68846-369d-43a8-ab30-d4fdcdb2b75b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.367916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 272/552] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:27.368854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d68846-369d-43a8-ab30-d4fdcdb2b75b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.377443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.378399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.378857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.380079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.381239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:27.381712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:27.384396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f14a-0f06-41e4-b840-5a8acee14cbf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:27.409999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 281/553] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:27.412318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.412616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.412901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.436302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:27.438690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:27.473394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.500040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d68846-369d-43a8-ab30-d4fdcdb2b75b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.526433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 282/554] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:27.527537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.546575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.547963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.548405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.549135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.550047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:27.550752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:27.552713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:27.564787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 273/555] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:27.566573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.604886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.649929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724b3b2-431c-4525-bb62-4d1271daeb89 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.687958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.688616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.688907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.689188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.707306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927aea-5c64-4471-97a7-1eb26a00be7c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.721910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.748797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.750133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.750423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.750706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.761495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:27.770167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:27.771126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:27.772009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 274/556] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:27.772709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.775945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:27.778621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:27.865180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:27.879381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32312c-a07d-4397-8837-bf2bc6cb218e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:27.900303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 283/557] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:27.900303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.917731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:27.918628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.918982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.919784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.931390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:27.932842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-735e8423-0a1a-42ce-ae8c-0bc057f7c83e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:27.952030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:27.954009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:27.954900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:27.954900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:27.954900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:27.954900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:27.954900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 275/558] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:27.955174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e62da-1e41-45e2-a9da-ccf1976b1425 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:27.955702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e62da-1e41-45e2-a9da-ccf1976b1425 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:27.955702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e62da-1e41-45e2-a9da-ccf1976b1425 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:27.956996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 276/559] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:27.957533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e62da-1e41-45e2-a9da-ccf1976b1425 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:27.977569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.018429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74ce21-9d0a-4d36-8ffa-4e0b8d4b3adf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.053501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.054840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.055307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.055739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.056648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.056960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.058335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.067070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 284/560] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.067812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.074944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e62da-1e41-45e2-a9da-ccf1976b1425 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.112207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.113005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.113291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.113570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.123047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.126182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.128729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-740431ad-4c68-4851-8b7e-48a581b4df7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:28.148625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:28.149535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 277/561] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.155409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.171757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98d274-41d4-4275-80d8-6a32c3becbe5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.172347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:28.173536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:28.199049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.208778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.209474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.209765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.210052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.210791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.211105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.212507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.229023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 285/562] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.229023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.259729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 278/563] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:28.260861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.342317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9473de-a896-4559-9de7-cf05de70b5b1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.371872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee9db-8812-4526-aecb-13cfd140a6eb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.378008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.378800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.379084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.379391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.389287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.390801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.394039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.405808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.406403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.406692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.406966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.407732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.408040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:28.408503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:28.409471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:28.410482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:28.410482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:28.410482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:28.410482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:28.410775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.410848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 286/564] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:28.410848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.421909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 279/565] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.422536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.457258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.517719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bd41c-742e-44ce-a18a-a3b454625e17 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.519273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786abaa-f1c2-4376-a6af-9f4563052d74 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.552051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.552379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.552838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.553123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.553212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.553400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.553499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.553707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.560395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.561863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.563612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3c3a2d6d-744a-4138-953d-77b30ae21082 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:28.571362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:28.572331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 280/566] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:28.578272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d94d6-56f2-45d7-a935-7ee0b2a40301 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.579011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d94d6-56f2-45d7-a935-7ee0b2a40301 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.579011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d94d6-56f2-45d7-a935-7ee0b2a40301 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.579011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 281/567] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:28.579011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d94d6-56f2-45d7-a935-7ee0b2a40301 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:28.580140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:28.581229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 287/568] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:28.582541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.625335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.677405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.681293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78e351-dfc7-468e-8f17-842d072984eb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.683558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d94d6-56f2-45d7-a935-7ee0b2a40301 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.715781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.715973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.716567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.716660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.716854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.716959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.717131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.717220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.717849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.718681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.720074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.724421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.725843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.731144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 282/569] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.731947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d36365c4-bcf4-4317-a0f4-3e2eccaef41c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:28.740196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:28.741275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 288/570] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.746791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.747464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.759634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:28.760842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:28.826439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 289/571] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:28.827263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.842942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f757a-fd3d-463c-9d87-f1db93a39c4e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.845161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.877328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.877993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.878280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.878553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.879336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.879647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.881011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.888672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 283/572] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.889431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.912945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:28.925698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969317da-aa5c-4641-a318-d0a65fadf91f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:28.959686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:28.960362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:28.960653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:28.960935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:28.961783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:28.962103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:28.963560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:28.980871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 290/573] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:28.981651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:28.991713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dddb0b-bd28-4a1d-8d6f-2003321a7f53 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.024602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.025489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.025768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.026040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.034084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.035565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:29.049658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.050779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 284/574] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.051772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.077877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ce139-25bb-437b-89b6-fa37d3ec6699 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.080634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.110924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.111614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.111898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.112173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.112964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.113279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.114687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.125869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 291/575] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.126793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.135111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.161428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2caaf8f-425d-4059-8d55-8d59702bd337 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.194450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.195267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.195557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.195834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.206181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:29.213959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 285/576] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.216179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.230294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca67d-c385-43ad-8cf3-bc2f13365ad3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.268653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.269468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.269750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.270022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.280237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.281693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:29.298486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.299445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.300545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:29.300545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:29.300545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.300545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:29.300545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 292/577] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.300861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.307004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.323129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499bc3b-8783-4685-964b-d29bec35281c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.355838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.356549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.356835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.357123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.357412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.368274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.369893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:29.387062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.388141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.389226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:29.389226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:29.389226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.389226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:29.393208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 286/578] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.394023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.399741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f76798-fcae-4968-b836-9fd64edde717 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.431889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.432552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.432836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.433114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.433955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.434268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.435721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.453533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 293/579] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.454224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.495478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931040e7-ea04-4246-855a-ae4743b40d77 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.529047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.529695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.529979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.530252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.531127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=44f05b05c1c94c44aa499f224ac20648) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.531464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.532885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.535655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.539574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 287/580] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/role_assignments?user.id=44f05b05c1c94c44aa499f224ac20648&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.540556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.550845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950742c-9ae2-4e16-b438-ba7be8b6f644 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.569903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.591572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.592268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.592559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.592828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.593657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.593970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.595426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.605269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 294/581] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.606160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.664331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9939-3d28-4fcc-bc6b-f7b9dacca9f7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.703920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.704925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.705421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.705847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.706749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.707206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.708775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.721093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5636e-1874-4367-9906-a926b6e92c40 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.721493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 288/582] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.722306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.759814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.760409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.760700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.760981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.761709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:29.762020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.763700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.775342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 295/583] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:29.776137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.786807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.820733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:29.834547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee5dd-9ce8-449a-809a-d7af7b6d8156 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.872092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.872903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.873185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.873465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.883554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6e176-43d3-45dc-8f8d-df87d95f8e64 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:29.884593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.886055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:29.904074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.905162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.906177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:29.906177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:29.906177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.906177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:29.906177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 289/584] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.906599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:29.928088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:29.928887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:29.929174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:29.929446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:29.944112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:29.945580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:29.969987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:29.971081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:29.972212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:29.972212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:29.972212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:29.972212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:29.975576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 296/585] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:29.976670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.012773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f27be5-7914-4d39-bd59-b981987f0087 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.026724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.050030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.050829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.051113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.051414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.058440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.060004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.074924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:30.075069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:30.076154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 290/586] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:30.077918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.089347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020d1b9-db16-4b9c-b200-c09a1c2dc14d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.125362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.126282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.126668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.127032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.134694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.136376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:30.150745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:30.151751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:30.152686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 297/587] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:30.153249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.187972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72ca9f-862f-4d57-9a2b-14a07265e378 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.226653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.227390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.227688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.227969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.228704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.229018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.230509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.248675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 291/588] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:30.248675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.259013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18893-3964-4f81-90e7-74558229eaf3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.280261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.296939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.297733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.298130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.298497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.299259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.299580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.300941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.312511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 298/589] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:30.313287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.326027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.362994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe80bb2-63ef-4b65-a90a-97b588e8ae3e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.399256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.400125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.400685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.400973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.401821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.402137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.403741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.413097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385be88-7c14-4171-953f-b35ee1746b23 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.422375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 292/590] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:30.423350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.453399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.454067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.454372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.454636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.455463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.455781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.457235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.470273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 299/591] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:30.471069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.525667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.536958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049344d-6f71-4551-b3ec-c3e88e03dce4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.544926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.575155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.575986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.576274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.576552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.579098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed8e13-eced-4337-9fe3-52f1eab86d91 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.584032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.585475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:30.600139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:30.601048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:30.601958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 293/592] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:30.602461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.619544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.620368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.620659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.620934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.628936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.630412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:30.646328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:30.647390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 300/593] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:30.648348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.715735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6451be1-c341-46d6-9ccc-9c2b105de29d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.749901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.750695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.750984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.751298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.755388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a92bf9-970d-4710-9085-94b022df74d2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.762266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.763759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:30.781297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:30.782189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:30.783068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:30.783068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:30.783068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:30.783068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:30.783068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 294/594] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:30.783380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.785988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.791557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:30.798099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.798990 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.799306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.799583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.814812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=aff09cb4b1ff42c0a011f8e09a3c6c38, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.816331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.839250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 301/595] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:30 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:30.839250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.916267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58083b65-788b-449b-aa67-4fb80298408c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.951591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb9f04-8a16-414f-b750-573cc8f75a27 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:30.968979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.969943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.970300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.970586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.971787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.972499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:30.974132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:30.991064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:30.991919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:30.992475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 295/596] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:30.993485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:30.997303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:30.997711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:30.998701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:30.999091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.001203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.024352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 302/597] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:31.025355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.070295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.071871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.126321 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d1901-c54c-4049-acbe-4db81572180a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.144121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef9b40-622d-4816-8fb2-ed4b24b00c1d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.170596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.172106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.172715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.173011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.174036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:31.174359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.176495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.180264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.181227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.181580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.181924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/f11ea126550c4c4a8892522e01ed149a/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.191823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 296/598] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:31.192752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.199064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=f11ea126550c4c4a8892522e01ed149a, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.200632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.223445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 303/599] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:31 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/f11ea126550c4c4a8892522e01ed149a/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:31.224715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.317715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d156ed-1125-4c3f-9f23-18389668c496 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.343239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.357049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.358201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.359185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.359499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.372286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.379252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:31.393935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:31.395099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:31.396218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:31.396218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:31.396218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:31.396218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:31.396218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 297/600] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:31.401643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.426061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.454220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea4706-5562-4dd0-8881-82198328bdf0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.502397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.503202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.503510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.503792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.510782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.512263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:31.526042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:31.528365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:31.530973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 304/601] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:31.539237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.547686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645c685-322b-4f06-aa5b-74098f732adb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.584377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.585135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.585425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.585757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.586504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:31.586822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.588251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.603140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 298/602] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:31.609297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.641451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697e4a2-3ba9-40b6-8f09-3fed22d946ef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.645156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.677902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.678568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.678849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.679124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.680504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:31.681785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.683244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5189485-d62b-4693-8802-781ccbdcea18 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.701421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 305/603] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:31.702891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81717-ffd9-4c2f-b922-b4943e4e7377 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.703180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81717-ffd9-4c2f-b922-b4943e4e7377 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.703501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad81717-ffd9-4c2f-b922-b4943e4e7377 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.705543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 306/604] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:31.706376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81717-ffd9-4c2f-b922-b4943e4e7377 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.717617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:31.722085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4ff40-94e0-4969-a9c5-bb1341266cac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.760994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.761869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.762156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.762433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.770488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.772323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-71e25f31-74f3-4453-a6ff-8f4eb02dc475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:31.788055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:31.790090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 299/605] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.792226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.793612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.810429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:31.811999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:31.812371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad81717-ffd9-4c2f-b922-b4943e4e7377 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:31.847687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:31.848572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:31.848854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:31.849128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/1c6dce92d6724fbf878faba138471bb8/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:31.863995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=1c6dce92d6724fbf878faba138471bb8, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:31.865367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:31.887879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 307/606] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:31 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/1c6dce92d6724fbf878faba138471bb8/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:31.888895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.891891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 300/607] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:31.914268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:31.946506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:32.021375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:32.055404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828772f-949e-4ba7-a3c6-8636be862574 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.073437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232b25c-5ca6-477d-9108-e0a858f9ab78 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.094817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.095678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.095968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.096249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.108553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:32.113971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.114695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.114921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.115272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.116024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:32.116334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:32.117291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:32.118226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:32.119151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:32.119842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:32.119925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 301/608] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:32.124511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:32.132905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 308/609] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:32.138713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:32.263346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead862a-e1c5-429e-8a76-8e0c799000c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.293752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.295054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.295445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.295792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/edbb85bda10d45bf8f66c6ee0561c2a2/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.296817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:32.314960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=edbb85bda10d45bf8f66c6ee0561c2a2, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:32.316437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:32.338787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 309/610] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:32 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/edbb85bda10d45bf8f66c6ee0561c2a2/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:12:32.347916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:32.365330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714db958-d479-412d-b762-9803eeb4e697 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.414226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.414899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.415187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.415489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.416264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:32.416574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:32.418004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:32.430659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 302/611] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:32.436405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:32.564097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35d8b0-2e7c-4f03-b3e5-fee6a795d0b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.600429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.601231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.601520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.601797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.612634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:32.614097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:32.635523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:32.636424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:32.637321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:32.637321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:32.637321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:32.637321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:32.637321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 310/612] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:32.642682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:32.779923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6183e-3855-4597-9153-20a87e6254e7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:32.830789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:32.831504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:32.831794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:32.832078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:32.832819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:32.833134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:32.834519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:32.853537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 303/613] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:32.858984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:32.897508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.005913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6211dee-1399-42d5-a128-3738e5d38dcd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.048767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.049566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.049855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.050131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.060225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:33.062176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c270b66c-85ef-455e-905c-715cbd7aea59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:33.076663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:33.078209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:33.079169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 311/614] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:33.084530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.099786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.221202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ec27-d3b1-4242-b38c-d9cee789e3e9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.273106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.273788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.274070 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.274351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.275206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:33.275541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:33.276971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:33.305677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 304/615] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:33.309599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8df3-dece-4a1b-a0b7-e2ba8eb7f8b7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.310055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8df3-dece-4a1b-a0b7-e2ba8eb7f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.310221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8df3-dece-4a1b-a0b7-e2ba8eb7f8b7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.312230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 312/616] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:33.313040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.321454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.321740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.322019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.353998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:33.355434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:33.383590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.451140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfcc59-1b53-4a3c-bd7d-fa8203beb663 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.487066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.487816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.488110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.488391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.498958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:33.501003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:33.518923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:33.519879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:33.520787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:33.520787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:33.520787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:33.520787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:33.520787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 305/617] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:33.525482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.527696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 313/618] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:33.564703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.583561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.641445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7f34b-fa43-4d7b-8343-d7bc6233862f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.658255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.677974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.678649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.678937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.679236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.679975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:33.680287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:33.681669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:33.694580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 306/619] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:33.703623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.794662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a984081-0606-4372-9c84-4915ab68da80 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.809042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e9a86-f627-4f96-9419-8600d93b77c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:33.846278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.847136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.847456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.847782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.855190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:33.856793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:33.865270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:33.866361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:33.866872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:33.867384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:33.872020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:33.872916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:33.873824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 307/620] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:33.878662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.895662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:33.911246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:33.912449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:33.913487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:33.916125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 314/621] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:33.918930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:33.922462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:33.996574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc004c7-8eea-4b1a-a074-47cb81a52719 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.032402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.033229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.033612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.033992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.034889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:34.035331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.036893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f035c6-f2b7-4ed7-a319-d79e53602d8a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.044455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.050620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 308/622] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:34.056660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947f33-854a-45fe-9335-31026943967e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.056660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947f33-854a-45fe-9335-31026943967e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.056660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23947f33-854a-45fe-9335-31026943967e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.056660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 309/623] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:34.058125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947f33-854a-45fe-9335-31026943967e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.090619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ac70a-fa8a-41ac-bae5-f860673f79a1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.136270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.137543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.138606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.139269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.141022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:34.142878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.144710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c109ff-7955-4cce-ab20-47cfe9e36109 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.154132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 315/624] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:34.156363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5e5a-b31b-43bb-a43f-9cb21405a62e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.156874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5e5a-b31b-43bb-a43f-9cb21405a62e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.157385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c5e5a-b31b-43bb-a43f-9cb21405a62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.176733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-558c5e5a-b31b-43bb-a43f-9cb21405a62e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:34.178003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.179329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-558c5e5a-b31b-43bb-a43f-9cb21405a62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:34.186756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23947f33-854a-45fe-9335-31026943967e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.225042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.226276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.226786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.227361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/44f05b05c1c94c44aa499f224ac20648/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.242748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=44f05b05c1c94c44aa499f224ac20648, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.244609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.259682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 310/625] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:34 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/44f05b05c1c94c44aa499f224ac20648/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:12:34.260482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.270981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 316/626] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:34.272597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947cdf-1c22-482a-a331-3ce3886c6e9a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.272900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947cdf-1c22-482a-a331-3ce3886c6e9a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.273190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4947cdf-1c22-482a-a331-3ce3886c6e9a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.274921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 317/627] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:34.280251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.281159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.281451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.311738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:34.312888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:34.378948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.379723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483df15c-e18a-402d-94c0-dd50aa3b62b7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.392409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 318/628] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:34.393282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.417850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.418667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.418955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.419534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.431099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.432642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.441337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:34.454830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:34.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 311/629] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:34.458985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.526922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03ed20-f153-4be0-973b-7cab05b86f37 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.564055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.565109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.565232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.565426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.576801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:34.578824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e8e2e-efc4-4f0a-a1f0-aa7b6c9b5ae7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:34.585369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:34.586596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 319/630] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:34.587864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.620020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.621098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.621533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.621971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.634110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:34.643415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:34.644449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:34.647118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:34.648299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:34.649353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 312/631] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:34.650309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.694504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1be4-9ed5-4386-a7ee-ef42995b7fb3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.715687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.732629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.733299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.733630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.733911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.734661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:34.734972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.736529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.754147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 320/632] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:34.754147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.769554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e2b1b-a8bc-4f6d-8e58-bc2b6f3779a7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.806761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.807744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.808255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.808679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.809701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:34.810208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.812294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.821874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 313/633] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:34.823037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.857004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088a5e6-a362-4539-8d28-91550107cf24 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.871741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.897133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.897741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.898037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.898319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.899080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:34.899543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.901021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:34.910061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 321/634] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:34.910874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:34.944108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4483-ca42-449e-a4ce-8ace35e016b3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:34.966088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:34.982541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:34.984604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:34.984903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:34.985184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:34.995194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:34.997041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:35.012001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.013002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:35.013999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:35.017740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 314/635] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.018495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.021892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1e8ab-fd0f-4eed-99f5-205f821990bf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.057831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.058629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.058916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.059194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.072655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.074050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-453dd300-bc93-4bb7-9f1c-6909067cf05d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:35.091797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.092752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.093690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:35.093690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:35.093690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:35.093690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:35.093690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 322/636] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.094736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf292cc-7182-4bae-838e-348c4fe46a67 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.095024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf292cc-7182-4bae-838e-348c4fe46a67 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.095327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf292cc-7182-4bae-838e-348c4fe46a67 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.096853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 323/637] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:35.097721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf292cc-7182-4bae-838e-348c4fe46a67 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.128756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.129117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db5fcd-d674-4d4f-98a2-9d0cb2ad20eb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.164644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.165447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.165744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.166205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.177726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.179185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:35.198575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.199489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:35.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:35.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:35.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:35.200465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 315/638] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.200746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.210688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf292cc-7182-4bae-838e-348c4fe46a67 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.235688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.250393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.250761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.251047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.251369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.252229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:35.252544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.253981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967bacd-27cd-45c5-a380-8a535b0ffe42 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.271558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 324/639] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:35.277300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.277606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.277893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.294954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:35.296772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:35.315925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d537eb-2433-4245-a373-dfa2356d836a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.351177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.351808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.352087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.352390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.353098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:35.353412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.354774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.364371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 325/640] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:35.365199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.383510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 316/641] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:35.384390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.385221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.479164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a4fd-4199-49db-a1d3-2f08521b091f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.484319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.494795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd53e8-b9c6-46e4-80b6-2672cb19244e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.517509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.518121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.518413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.518687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.519454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:35.519764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.521138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.531818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.532477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.532767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.537556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.537675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 326/642] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:35.537675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.610702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.612138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.618636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.652313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74b5e-4168-4dc2-9346-2aaed2fec276 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.687849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.688642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.688925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.689206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.696860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.698676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.706983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:35.714416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.715457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 327/643] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.716411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.730167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 317/644] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:35.730167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.828313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ed18e-5704-4ea3-a9a8-47d93883f710 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.829314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7114-b2f8-403c-a44d-1babc408b39c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:35.865663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.865849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:35.866396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.866518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:35.866679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.866771 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:35.866959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.867060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:35.869249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:35.874826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:35.876292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:35.880566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:35.888324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.889265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:35.890804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 328/645] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.892169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:35.896144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:35.897894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 15:12:35.900177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:35.902112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 318/646] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:35.902710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:35.958557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.002170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbaea-bcd1-472f-b1e8-8aeea7f6d8a8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.007867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56f1fa-6f1e-4224-94b7-24ebd74f42bf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.037136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.037981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.038264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.038652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.039533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:36.039848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.041371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.043818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.044949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.045304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.045655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.056051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.057504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.060086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 329/647] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:36.060951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:36.080724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:36.081830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 319/648] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:36.087626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.124775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.172175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002138c0-fe42-43b1-a570-6a1fb8f003d9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.191629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19533615-5ced-4eaa-8f50-959a98588c5e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.194103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.207950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.208630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.208921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.209201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.209959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:36.210271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.211711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.219836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 330/649] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:36.220644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.233249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.234345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.234818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.235298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.236318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:36.236884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.238747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.259521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 320/650] 10.222.0.46 () {56 vars in 1139 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:36.264501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.334138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d96a72-3d57-458d-bca7-5e1ac9b191a7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.347924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.369598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.370316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.370574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.370857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.377728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5bfac6-37fb-41fc-93a1-cc68e425b44e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.409864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.410525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.410810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.411090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.411841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:36.412164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.413550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.425298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 321/651] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:36.426088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.443172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.444649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.450302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.524184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c84c-b9aa-4212-9800-2d2e4170016e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.542515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 331/652] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:36.543365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.558737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.559572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.559868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.560004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.560145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.571639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.573080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0b0d4784-0953-4dde-af67-af5cb7912bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:36.589940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:36.590873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:36.591989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:36.591989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:36.591989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.591989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:36.591989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 322/653] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:36.592549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f86ff-9555-4779-ba4d-3d98ed092e56 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.592549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f86ff-9555-4779-ba4d-3d98ed092e56 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.592855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f86ff-9555-4779-ba4d-3d98ed092e56 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.594188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 323/654] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:36.594751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f86ff-9555-4779-ba4d-3d98ed092e56 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.648536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad4460-9ada-4540-ae94-2769c1aed203 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.670110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.682867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.683545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.684148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.684148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.696741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f86ff-9555-4779-ba4d-3d98ed092e56 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.729829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.730618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.730907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.731190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.738373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.739865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.751515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.752957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:36.753509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:36.754460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:36.755411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 324/655] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:36.755927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.771070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.854589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88575e2a-dbfb-4789-a2f2-bd204a067091 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.855056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 332/656] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:36.855909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:36.889419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.890212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.890500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.890783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.892718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.902447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.903891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bc7cc776-512e-40cc-a700-0e333347d0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:36.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:36.923201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:36.924198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:36.924198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:36.924198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:36.924198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:36.927357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 325/657] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:36.928846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.929142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.929430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.946576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:36.947783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:36.953324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d921bf-4153-47a8-8bdc-81ab40d525a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:36.967415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:36.988794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:36.989463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:36.989752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:36.990035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:36.990766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:36.991079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:36.992476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.003767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 333/658] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.004545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.013565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 326/659] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:37.014383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.104871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7dcab-1722-4240-804a-141411493327 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.120107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca9410-a129-41d2-b429-fbd92e569310 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.135980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.137861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.138527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.138977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.139294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.140632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.140945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.142375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.154126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 334/660] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.154842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.156363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.157168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.157459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.157744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.168050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.169883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:37.186985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:37.188080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 327/661] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:37.189006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.189940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.257329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e71cb2-a82a-461f-907d-2f56f5cd1735 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.286728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42487f-1e42-478e-8e25-0d6e697b29a9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.290991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.291687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.291975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.292252 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.292973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.293284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.294917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.306065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 335/662] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.306920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.324641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.325448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.325736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.326016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.334071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.335537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:37.349965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:37.350956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 328/663] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:37.352039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.379944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.404781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.412576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4489e-576f-40a8-8609-f86bf147ddd0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.445705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.446492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.446773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.447044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.452777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb05c2a-a867-4e35-b5a7-fd79800aa801 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.458159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:37.465784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:37.466677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:37.467824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 336/664] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:37.468757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.489803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.490572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.490918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.491286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.492205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.492586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.494073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.500334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 329/665] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.501152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.577977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cdc39-1ef5-4d99-838b-f14f7f35f46d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.604408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.615331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f3738-d5f6-4c4e-962b-406b40eddfa6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.616813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.617577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.617938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.618305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.619136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.619494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.620920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.635238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 337/666] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.636101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.655611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.661580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.662584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.662866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.663139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.671306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.672860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:37.689805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:37.690770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:37.691737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 330/667] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:37.693130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.765549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2620df-cfdc-4d05-96b8-e5003a11cc59 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.800908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.801585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.801871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.802158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.803005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.803357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.804790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.805557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06cc39-66d4-47a2-bfb4-36077e8b2f48 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.827043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 338/668] 10.222.0.46 () {56 vars in 1137 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.827043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.842983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.843731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.844072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.844358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.845112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:37.845420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.846798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:37.860078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 331/669] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:37.860078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:37.877693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.909392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:37.944120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e7d73-a18c-4389-814a-51ed0be0bb48 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.968908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcca61-f301-4037-9eb7-8e733c3ae9ac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:37.980450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:37.981232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:37.981518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:37.981800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:37.989351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:37.990782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.004880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.005552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.005840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:38.005940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:38.006769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:38.008331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:38.008789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.008789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.008932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 339/670] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:38.008932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.011709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.013227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.031486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 332/671] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.031486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.113035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.117708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7925a3-715a-4b53-a3cf-39404e704a41 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.141643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c892d-8b81-4ad5-8999-f2c0468dfba0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.148681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.154748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.155679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.155969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.156254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.177038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.177897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.178183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.178455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.189413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.190857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:38.210094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:38.211034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:38.211997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:38.211997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:38.211997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:38.211997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:38.211997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 333/672] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:38.212701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.233447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.235453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.326714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d85338-e2e9-417e-9795-fc109cc0e086 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.344930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 340/673] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:38.349949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.353717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.363374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.364049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.364337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.364616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.365414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.365729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.367162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.379210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 334/674] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.380014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.389040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.461978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d66da8-649f-4d37-9ce5-3a26b25bf59a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.486113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20cb8-f444-4873-8bb7-58501d54ae2c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.497365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.498027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.498316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.498600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.499474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=f11ea126550c4c4a8892522e01ed149a) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.499793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.501221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.507061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 341/675] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/role_assignments?user.id=f11ea126550c4c4a8892522e01ed149a&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.507860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.537871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.538972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.539782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.540219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.541695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.542451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.544634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.566910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 335/676] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.566910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.624436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.632879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eee43-dc66-4b35-8e08-c8139bff7461 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.633472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.674231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.675084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.675431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.675786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.688116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.689588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.697779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c36f3-99b2-4f5a-9852-d6c93964a741 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:38.709875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:38.710794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:38.711832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:38.711832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:38.711832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:38.711832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:38.711832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 342/677] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:38.712167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.741858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.742530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.742817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.743101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.744016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=1c6dce92d6724fbf878faba138471bb8) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.744720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.746262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.752817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 336/678] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/role_assignments?user.id=1c6dce92d6724fbf878faba138471bb8&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.753718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.845138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597599ba-f72d-4848-8438-969122d08b74 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.867974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f9a7-7bea-4c52-b873-4234cfeabc06 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:38.879363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:38.890852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.892308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.892590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.892862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.904454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.906551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.912213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:38.912900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:38.913186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:38.913461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:38.914203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:38.914513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:38.915937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:38.926777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:38.927882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:38.929021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:38.931339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 343/679] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:38.935762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 337/680] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:38.937388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.939894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:38.950517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.046868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1bb0-47d8-4b7e-8b74-9390b41aeda5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.050087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9077cfd-95f2-48ce-a6a1-7e767ef76d06 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.083417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.084734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.085210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.085594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.085909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.086271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.086552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.086830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.087717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:39.088032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.089469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.095542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.097250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.108817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 344/681] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:39.112895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:39.114037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:39.115176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:39.115881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.117634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 338/682] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:39.129310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.171620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.191866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.245319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e1d53-923a-45e9-ba8c-28f1ca26548f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.263480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd97e-5289-4571-8cc8-5549a04f6c5f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.299745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.300344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.300634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.300913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.301723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:39.302039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.303540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.317674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 339/683] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:39.323460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.332125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.332843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.333126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.333400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.424826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.433379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abf94e-2241-4fa0-bb66-c39cb36aa5b7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.490259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.491759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911ccc3-e9fc-4fff-aa83-10d9b0821748 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.498124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.498805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.499757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.500044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.500898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=edbb85bda10d45bf8f66c6ee0561c2a2) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:39.501218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.502664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.516608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 340/684] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/role_assignments?user.id=edbb85bda10d45bf8f66c6ee0561c2a2&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:39.574056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.626189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 345/685] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:39.631682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.762002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81e1b-d190-4540-a2f1-e8e9a988cffd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:39.800601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:39.801415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.801701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.801978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.804746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461986e-cf1b-4e4c-82ff-998539ac0412 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.805033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461986e-cf1b-4e4c-82ff-998539ac0412 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.805309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461986e-cf1b-4e4c-82ff-998539ac0412 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.806801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 346/686] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:39.811999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:39.812290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:39.812570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:39.817145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:39.818609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:39.838572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:39.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:39.840483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:39.840483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:39.840483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:39.840483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:39.840483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 341/687] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:39.842117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:39.844848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:39.845856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.857727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:39.927614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 347/688] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:39.952530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:39.974788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c8713-1c5f-44dd-8616-bbe48ca69ece None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.023203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.024449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.024874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.025244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.026321 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:40.026697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.029085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.045092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 342/689] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:40.052783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.059570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:40.075583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031f7a-18f3-499d-982a-084d04e70122 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.126003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:40.140737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.142344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.142841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.143376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.161234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128157ae-d099-4c4f-83d9-bcd61f5cef67 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.164579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:40.182695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:40.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:40.184573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:40.187337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 348/690] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:40.193275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.195133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.196131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.196593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.197054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.204954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.206622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:40.221951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:40.222852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:40.223783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:40.226381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 343/691] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:40.231694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.314977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5aaf0-36f9-45dc-906d-4fc969a4ea8f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.349589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.350508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.350979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.351465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.352372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:40.352832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.354361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.364017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 349/692] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:40.369629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.395030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:40.478677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb49b38-66d9-41bd-924c-2b896e2da43c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.515816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.517469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.517976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.518457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.522624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ac0a6-fb6d-43cf-a21c-047e20692146 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.529608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.532623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:40.552621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:40.553568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:40.554483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:40.554483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:40.554483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:40.554483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:40.556831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 350/693] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:40.562430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.578218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:40.598335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.599400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.599903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.600387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.611905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:40.612506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.614182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.636309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 344/694] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:40.641847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.695648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:40.749391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5847d4-a758-42a8-8d38-32ac43cd107b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.791150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7f80d-a38e-4e4b-a309-8b4fc1c66c3f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:40.838035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.838704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.838987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.839283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.840026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:40.840336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.841695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.847564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:40.848223 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.848507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.848786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.849622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=44f05b05c1c94c44aa499f224ac20648) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:40.849938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:40.851375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d706ca-2c52-4244-bfc1-47b95afd6f8d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:40.866132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 351/695] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:40.868082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 345/696] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/role_assignments?user.id=44f05b05c1c94c44aa499f224ac20648&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:40.872326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:40.920560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea47f6-d97f-4d68-8436-d5dc74e2459e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.920952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea47f6-d97f-4d68-8436-d5dc74e2459e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.921154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea47f6-d97f-4d68-8436-d5dc74e2459e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.922691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 346/697] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:40.932885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:40.933259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:40.933460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:40.969039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:40.970146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:41.006040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2262a7c-7a1e-4201-80c3-472f8d0994b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.056959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.065476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.074495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.075377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.075661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.075937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.084930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.086425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.102318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 347/698] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:41.103564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:41.104481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:41.105405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 352/699] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:41.110180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.135473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.226260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad791f9-3378-46f2-8d05-6bfac5a8b2ef None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.268386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecbf36-93cc-47fa-a198-d22a4740e097 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.268740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.269600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.270037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.270467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.271514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:41.271831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.273403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.293189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 348/700] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:41.298795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.315592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.316893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.317181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.317459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.337010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.338491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.351606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:41.364504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:41.365440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:41.366331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:41.366331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:41.366331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:41.366331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:41.366331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 353/701] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:41.372570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.429078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a1bfb-27fd-4038-bfaa-4f29dceccc7f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.435006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.471075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.471791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.472080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.472355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.508097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b8abc-b019-44e9-898d-8b0018915e29 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.545347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.546259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.546597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.546883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.548149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:41.548745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.549474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.550122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2dc6a-8d5a-48f9-96f8-c78c9f24ee8d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.550913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d64ea-0f41-4def-92a9-f2232dc149a9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.565725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 354/702] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:41.567883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefba5c6-5f30-420d-8116-fbe499f24260 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.568173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefba5c6-5f30-420d-8116-fbe499f24260 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.568450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefba5c6-5f30-420d-8116-fbe499f24260 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.570052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 355/703] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:41.571666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefba5c6-5f30-420d-8116-fbe499f24260 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.595532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.692283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 349/704] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 15:12:41.692838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.692945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.693206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.695084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefba5c6-5f30-420d-8116-fbe499f24260 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.697174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.711103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:41.712368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:41.742455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.743716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.744443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.744925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.746507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.748485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:41.776768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 356/705] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:12:41 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:41.778121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.793264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 350/706] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:41.829110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:41.914444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea057f4-0f5b-4731-ab80-4b14731f554b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:41.960966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:41.974917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:41.976250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:41.976749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:41.977233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:41.990953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:41.992679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.003166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16b7f-6100-4aa9-85dd-0a2240584523 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:42.015257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:42.016172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:42.017072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:42.017072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:42.017072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:42.017072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:42.017072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 357/707] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:42.022573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.048640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.049626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.050010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.050390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] PATH_INFO: `/v3/services/network` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.051430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.051972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:42.053232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:42.070311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:42.071418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.072263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 351/708] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:41 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:42.077433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.136100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18d82a-a9c1-452d-b2ed-7c700437be44 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.179076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.179849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.180247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.180532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.181287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:42.181604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.183120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.197402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 358/709] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:42.202991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.212414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0621d-5400-46f3-931d-99813d17897f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.278424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:42.287621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.288303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.288594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.288877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.290531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] RBAC: Adding query filter params (name=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:42.291144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.294148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.306326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 352/710] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:42.311420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.335597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:42.374266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599e202-d7d3-4c53-b614-ea2ed05eb46e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.412472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.414149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.414878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.415381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.424567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.426672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5106d259-5dcd-481f-b8c9-cea1d57733aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:42.443449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:42.444402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:42.445302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 359/711] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:42.450747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ea05-785c-4ef3-8c58-c0a565553ba1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.451117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ea05-785c-4ef3-8c58-c0a565553ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.451684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ea05-785c-4ef3-8c58-c0a565553ba1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.457403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 360/712] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:42.457403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9ea05-785c-4ef3-8c58-c0a565553ba1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.466163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe220ec7-8838-4337-82e4-ecf16ae8dcec None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.504205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.504974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.505363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.505648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.506463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:42.506777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.508212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e022331-c960-45e1-813a-dd4ebeebaef4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.514269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 353/713] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:42.516160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.516453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.516738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.535255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:42.536522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:42.603126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:42.612036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9ea05-785c-4ef3-8c58-c0a565553ba1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.628436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 354/714] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:42.654799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:42.662806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.666644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.672615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.672900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.673182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.675865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:42.676191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.678921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.703365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 361/715] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:42.710878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.792494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcece-9f88-44f6-ba86-dee754c1c451 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.829392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.830986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.831510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.832038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.832904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.834520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.839307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896aa835-0ee8-4922-a278-c58556ea08bc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:42.847454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:42.848426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:42.849314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:42.852199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 355/716] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:42.857778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:42.875995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:42.876706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:42.877000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:42.877282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:42.878125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:42.878443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:42.879917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:42.886020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 362/717] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:42.943806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:43.022434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f23af-a044-4640-9587-0e646c27d60e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:43.068767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:43.069453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:43.069741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:43.070076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:43.070749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:43.071061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:43.072472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1778517-4fcf-4d70-9222-65c7740e9578 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:43.078940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 356/718] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:43.084123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:43.233978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:43.300929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19c7dc-13e7-49e1-99de-33c881a331d8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:43.347949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:43.348646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:43.348940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:43.349227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:43.349969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:43.350285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:43.351701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51257f-ac73-4292-b735-a3eab80a263e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:43.358231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 363/719] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:43.663945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94bb0-ab12-4a39-b2ea-0305bb5541a7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:43.664135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94bb0-ab12-4a39-b2ea-0305bb5541a7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:43.664428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94bb0-ab12-4a39-b2ea-0305bb5541a7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:43.666115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 357/720] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:43.671989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:43.672444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:43.672543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:43.696245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:43.697475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:43.811259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 364/721] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:43.856351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94bb0-ab12-4a39-b2ea-0305bb5541a7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.072166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94bb0-ab12-4a39-b2ea-0305bb5541a7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:44.114163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:44.145362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:44.146185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.146476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.146806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.179947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:44.195240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:44.196270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:44.197136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 358/722] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:44.204391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.261182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:44.317200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88736897-3ac6-4867-9057-f8b1bc49ea59 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:44.358875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:44.359511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.359805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.360088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.360853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:44.361166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:44.362543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb1a8e-5ca2-4da8-8a95-c6bf52f032c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:44.371302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 365/723] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:44.376633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.509965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61e3be-6ee1-4001-8f16-c75665a8b04f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:44.513102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e5ec-d0a7-400f-973c-3d4a78b33213 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.513400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e5ec-d0a7-400f-973c-3d4a78b33213 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.513685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5e5ec-d0a7-400f-973c-3d4a78b33213 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.515285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 366/724] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:44.522710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.522710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.522710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.542150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:44.544530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:44.562683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:44.563802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.565044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.565538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.587349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:44.589162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:44.617564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:44.618544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:44.619453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:44.619453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:44.619453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:44.619453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:44.622172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 359/725] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:44.628206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.659524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 367/726] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 15:12:44.674517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:44.686083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.762448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:44.823011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950a3c9-54ad-431e-9e4d-53edf09e7f1d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:44.883393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b06d7-b103-46d0-ab88-18a1657712a9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:44.890902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:44.891606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.891894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.892176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.892918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:44.893237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:44.894641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:44.925909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 360/727] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:44.931209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:44.963093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:44.963957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:44.964246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:44.964525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:44.992140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:45.000569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:45.002601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:45.004540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:45.011662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 368/728] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:45.017841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.032842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.043725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb10a7-651c-43b1-b462-1cf2de74e9a0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.081837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.083147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.083709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.084192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.092891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.094599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:45.110128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:45.111090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:45.112087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:45.114691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 361/729] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:45.120780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.183913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c4399-3917-4f6f-ae2d-f05358fc4901 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.187550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.260129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.261100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.261581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.262055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.262378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320126a-70d0-46e5-bc16-3206f8929b29 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.263252 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:45.263761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.265941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.274691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 369/730] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:45.280289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.322928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.328537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.329844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.330328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.332846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:45.333399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.336049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6bf70-2ceb-4510-9832-024abbc75c10 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.360839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 362/731] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:45.362382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7224cb1-530d-4c15-bb74-a743d50bccc1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.362838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7224cb1-530d-4c15-bb74-a743d50bccc1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.363338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7224cb1-530d-4c15-bb74-a743d50bccc1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.365126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 363/732] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:45.366059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7224cb1-530d-4c15-bb74-a743d50bccc1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.388435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224c13e-9e83-4a64-b935-9adabfe904e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.415616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.423866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.424753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.425103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.425446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.436938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.438449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-36d5fc50-60f8-4dbc-b830-19e4d494f6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:45.456178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:45.457092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:45.458002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:45.458002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:45.458002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:45.458002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:45.461927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 370/733] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:45.463035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4723-aaca-4bca-8ad1-383796d2eacb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.463350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4723-aaca-4bca-8ad1-383796d2eacb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.463631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab4723-aaca-4bca-8ad1-383796d2eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.479003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7224cb1-530d-4c15-bb74-a743d50bccc1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.480967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f6ab4723-aaca-4bca-8ad1-383796d2eacb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:45.482152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6ab4723-aaca-4bca-8ad1-383796d2eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:45.496156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.514863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.515564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.515849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.516126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.591473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.593003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9fecc-65fc-4a8f-91ef-88423c8811e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.604263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.671900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 371/734] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:45.673021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e4c-2772-44ac-b95a-fbc9faabbc41 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.673306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e4c-2772-44ac-b95a-fbc9faabbc41 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.673580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e4c-2772-44ac-b95a-fbc9faabbc41 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.679077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 372/735] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:45.679077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2259e4c-2772-44ac-b95a-fbc9faabbc41 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.717288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 364/736] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 15:12:45.717288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.717288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.717288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.738014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:45.739201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:45.786916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2259e4c-2772-44ac-b95a-fbc9faabbc41 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.809215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 365/737] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:45.809949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.843147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.843843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.844129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.844404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.845122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:45.845429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.846794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.852979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.859364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 373/738] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:45.864230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.868937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:45.917765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349ea0a-7028-4043-bdb4-54e07bdc869d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:45.955548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:45.956219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:45.956507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:45.956779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:45.957498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:45.958878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:45.973822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 366/739] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:12:45 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:45.974568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:45.977106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0eea-c859-4ca3-b824-c52a0b3556bf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.032504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.033321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.033610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.033890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.046555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.048131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:46.067236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:46.068248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:46.069143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:46.069143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:46.069143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:46.069143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:46.073435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 374/740] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:46.074269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.103205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.108170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e51184-664c-4614-a57f-c420e0760971 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.137791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.152624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.153467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.153753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.154037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.165883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3d3d51ca-b146-408d-9e6a-c7fe85f81eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:46.174025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:46.174957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:46.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 367/741] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:46.176877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4ce5-1770-442a-a68b-b1229a6478a1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.176946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4ce5-1770-442a-a68b-b1229a6478a1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.177504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4ce5-1770-442a-a68b-b1229a6478a1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.178566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 368/742] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:46.179174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a4ce5-1770-442a-a68b-b1229a6478a1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.204824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b58d8-2a0f-47ab-8670-6e6a6b9502ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.241902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.242663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.242922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.243197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.251357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.252784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-98401df8-14ba-479f-9a05-bda4b1f48dff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:46.268209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:46.269172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:46.270149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 375/743] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:46.270760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.270860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.271865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.292495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:46.294532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:46.299746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a4ce5-1770-442a-a68b-b1229a6478a1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.335406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.336324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.336612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.336893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.337626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:46.337943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.339353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.355917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 369/744] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:46.355917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.371264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 376/745] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:46 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 15:12:46.372119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.387242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.417922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.478475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d25cc-0826-4d8c-b875-4936db19334e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.482206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70c6c5-ed8a-4fb1-ba53-00ff08c257de None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.513823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.515098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.515421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.515985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.516636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:46.516947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.518366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.520609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.521292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.521581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.521864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.522623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:46.522936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.524380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.532518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 370/746] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:46.533311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.540744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 377/747] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:46.541547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.619577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.640469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.642737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17e59-87aa-418f-9f13-79c7fb57d890 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.643762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a37fa-71ff-4995-800f-fc6fed600558 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.679314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.679995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.680284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.680566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.680795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.681301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.681913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.682350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.682687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.682934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.691004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.693079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.697053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 378/748] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:12:46 2022] POST /identity/v3/services => generated 251 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:46.697862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:46.716713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:46.717924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:46.719014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:46.721273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 371/749] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:46.722316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.833272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec436d-0f69-4e0c-9638-3c6b14ad4d1e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.840963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7897b-8c0c-4919-b1f5-07b8e14a9aa4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:46.883472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.884723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.885166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.885444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.886771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:46.887690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:46.888200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:46.888679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:46.898318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.900326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:46.901975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:46.925091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:46.926022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:46.926917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:46.926917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:46.926917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:46.926917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:46.926917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 379/750] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:46.927660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:46.938988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:46.971460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:46.972945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.048861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd964d-8a32-42bc-9f07-c0042ea1b237 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.082851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 372/751] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:12:47.084093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.090325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.091020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.091338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.091615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.092485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:47.092809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.094855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.109940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 380/752] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:47.110641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.131467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.187647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.191872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2c9fa-818f-4349-9f45-c803f62b226f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.225373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ab7f3-efad-4462-9b23-24d9b6abd109 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.226805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.227576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.227933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.228277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.229459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:47.229459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.230861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.247704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 373/753] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:47.248632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.273090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.274335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.274764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.275211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.291120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.291120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:47.310585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:47.311577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:47.312452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:47.312452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:47.312452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:47.312452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:47.312452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 381/754] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:47.313503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.349301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.352091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0894bf-9a0a-4a32-943d-a12db5f11b19 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.386991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.387843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.388229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.388758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.389574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=44f05b05c1c94c44aa499f224ac20648) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:47.389990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.391567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.398256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 374/755] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/role_assignments?user.id=44f05b05c1c94c44aa499f224ac20648&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:47.398538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.433975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ec8d-998d-4117-a30f-8a745e051d90 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.457110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.482421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.484200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.484641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.485061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.497442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.506996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39890465-9672-4362-ae52-4a9db3de5a3c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.507390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:47.530977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:47.532204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:47.533402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:47.535882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 382/756] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:47.542672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.562364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.562775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.563064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.563383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.564121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:47.564493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.565819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.588068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 375/757] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:47.597296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.641286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.666532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e75df-90d4-4f28-a6e9-686161354c0e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.709642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.710351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.710778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.711187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.712097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:47.712416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.713857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.738634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 383/758] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:47.743526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.760013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf80d-14f3-4361-8c6a-8af780614904 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.788470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.811888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.812704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.812996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.813282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.820821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.822288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:47.837555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:47.838496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:47.839401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 376/759] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:47.845361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:47.861662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca62b13-e7b3-47e8-af50-69d1c43219cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:47.929112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:47.929888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:47.930178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:47.930459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:47.955961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:47.957422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:47.995623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:47.999653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:48.000535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:48.001475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:48.001475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:48.001475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:48.001475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:48.001475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 384/760] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:48.006364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.028920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b74-595c-4455-bca7-f70eebb1488d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.070461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.071173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.071489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.071769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.072761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:48.073079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.074535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b34ba3-2706-4bf3-9ffb-d43e2cb32f3c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.088661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 377/761] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:48.090024 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843e96-29e1-452f-b6b7-883d50ae8292 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.090318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843e96-29e1-452f-b6b7-883d50ae8292 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.090601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843e96-29e1-452f-b6b7-883d50ae8292 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.096064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 378/762] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:48.096857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3843e96-29e1-452f-b6b7-883d50ae8292 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.145324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:48.160048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290f834-56fc-4336-8e16-3d7c63f96ff1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.198184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.198787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.199078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.199403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.200139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:48.200558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.202603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016faa8-7f9a-4e3f-b99b-f7a5b5c41b26 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.217680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3843e96-29e1-452f-b6b7-883d50ae8292 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.219289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 385/763] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:48.220646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.220946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.221229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.243558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:48.244844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:48.261001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.261927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.262449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.262930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.264094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=cbf595eb1a0c44afb4d4f96413763631) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:48.265182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.266886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.279545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 379/764] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/role_assignments?user.id=cbf595eb1a0c44afb4d4f96413763631&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:48.281007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.312054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:48.383517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 386/765] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:48.412933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0053-ff6b-4cdb-bc12-c22e0cf00b6f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.428333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.453729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.454706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.454990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.455333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.467951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.469584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.589317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:48.611438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a1b57-cbc1-453d-8d00-47dcf0267600 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:48.620781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:48.621693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:48.622618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:48.627049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 380/766] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:48.633893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.675333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.678254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.678756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.680358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.681267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.683234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:48.706454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:48.707556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:48.708423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:48.712230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 387/767] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:48.720669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.775814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:48.845788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02563a-ba0b-40b0-a385-ad4dc3014101 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.918566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.920086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.921745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.922432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.924356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:48.925053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.927773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:48.934257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ad111-73c4-48b5-8c08-455c523809a9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:48.954467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 381/768] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:48.961626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:48.991185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:48.991963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:48.992316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:48.992659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:48.993442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:48.994338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:48.995952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.001886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 388/769] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:49.007055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.023579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:49.129808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc47737-262c-425b-aa3b-97b90c4a5e35 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.160417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:49.165524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0e8b6-132c-4e2e-8ace-9237c15c31c5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.169642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.170791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.171295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.171789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.174408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=aff09cb4b1ff42c0a011f8e09a3c6c38, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.176455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef716ade-34d3-4b16-a0c0-e79d88fed8ae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.193725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 382/770] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:48 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:49.195482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b8544-921d-4bf6-a7f7-4b3097f24a9a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.196019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b8544-921d-4bf6-a7f7-4b3097f24a9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.196560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b8544-921d-4bf6-a7f7-4b3097f24a9a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.198098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 383/771] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:49.212575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43433b-3085-4904-925e-aa18c531005f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.213176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43433b-3085-4904-925e-aa18c531005f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.213679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43433b-3085-4904-925e-aa18c531005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.218645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.224002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.228388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.228983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.229954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:49.230488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.232214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.245491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ad43433b-3085-4904-925e-aa18c531005f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:49.246568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 389/772] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:49.248195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ad43433b-3085-4904-925e-aa18c531005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:49.253320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.349984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 384/773] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:12:49.392421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43433b-3085-4904-925e-aa18c531005f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.406046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:49.448458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf969-c884-4986-91ff-57bf0d9c8367 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.493580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.494659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.495142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.495846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.496939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] RBAC: Adding query filter params (interface=public, service_id=271d27fa8dfa4ab49de9f8d27bc2842a, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:49.497471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.499308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.502447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43433b-3085-4904-925e-aa18c531005f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.509713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 390/774] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/endpoints?service_id=271d27fa8dfa4ab49de9f8d27bc2842a&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:49.515466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.519711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:49.538213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.539028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.539391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.539666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.540335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.541758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:49.556162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:49.557989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:49.559683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 385/775] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:49.568424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.717106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15659eed-fc55-427f-9f5d-02887ba142c2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.758885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680f328-3e72-4fc4-9657-45baa3575a05 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.786041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.786773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.787061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.787364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.790108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.791507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.798859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:49.805605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 391/776] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:49.818690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:49.819404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.819689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.819966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.820701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:49.821011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:49.822426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d298976-4d0a-42a8-8678-4560d9db9eef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:49.829137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 386/777] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:49.835033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:49.940102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689609f-a9d6-456f-9b51-2b7c76a65efc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.940436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689609f-a9d6-456f-9b51-2b7c76a65efc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.940662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689609f-a9d6-456f-9b51-2b7c76a65efc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.942305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 387/778] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:49.948669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:49.949364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:49.949839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:49.959880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d5c26-4a0a-4d8e-980b-d49f64c8c245 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:49.991153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:49.992723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:50.004026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.004969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.005434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.005713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.006446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:50.006756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:50.008179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:50.019311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 392/779] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:50.020496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.053484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:50.087264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 388/780] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:12:50.092755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:50.116469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.137914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7564ee-746a-455c-8f83-3fb0479316b8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:50.174888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.175607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.175894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.176170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.177398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] RBAC: Adding query filter params (interface=public, service_id=5086b6aebee045e8be6c5eca050d160e, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:50.177729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:50.179280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:50.185100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 393/781] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/endpoints?service_id=5086b6aebee045e8be6c5eca050d160e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:50.191184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.307054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364e738-257e-4389-9529-cde23c4dc7c7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:50.344098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.344885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.345281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.345572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.346277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:50.347820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:50.350772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354e8ff-7e23-4fc2-aa15-fe93aa1c3e68 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:50.353631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 394/782] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:50.354203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:50.405520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.406328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.406621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.406903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.427634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:50.435421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:50.436961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:50.437791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 389/783] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:50.444798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.597261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a1067-bb0f-4dd2-bc2f-4c5d35ffcc13 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:50.637577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.638533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.638963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.639926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.640823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:50.641277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:50.642778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004e760-a8fd-4193-8e8c-78744193d82d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:50.652149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 395/784] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:50.658363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.689867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e37c4-5743-4e2e-8434-1152f6365cfe None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.690508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e37c4-5743-4e2e-8434-1152f6365cfe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.690795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e37c4-5743-4e2e-8434-1152f6365cfe None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.692222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 396/785] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:12:50.698349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.698637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.698913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.718694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:50.731939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:50.733161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:50.826065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:50.841371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5742f4-2ac1-4402-ba45-684fd9c573c6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:50.848615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 397/786] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:50.887941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:50.889085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:50.890092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:50.890567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:50.908190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:50.909870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:50.920239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:50.933225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:50.934131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:50.935019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:50.935019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:50.935019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:50.935019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:50.938169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 390/787] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:50.943940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.053744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceace79d-fc8c-47c4-968a-65bb2f8ed7ce None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.090906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.091858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.092146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.092464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.103494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:51.111565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:51.112523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:51.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 398/788] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:51.124719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.140876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.148157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada2bd6-5c8e-47c2-8439-235a88d6bfef None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.194484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.195635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.196145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.196619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.197545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:51.198108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.199208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.200810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.215657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 391/789] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:51.221507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.322609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858c5e0-9a59-4e0b-b47d-4d36f39ab01c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.385407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.386004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.386292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.386571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.387353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:51.387731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.389022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.395065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556397b-d863-4808-a79f-d3d5e6697d76 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.411441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 399/790] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:51.422787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.447413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.448659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.449139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.449707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.462752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.464502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:51.494585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:51.495603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:51.496551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:51.502069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 392/791] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:51.503810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.523520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.559519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a35252-cd69-461f-822a-5ef0b9b1912b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.571658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.621255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.622058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.622341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.622614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.641057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.642454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:51.678763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:51.679761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:51.680650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:51.680650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:51.680650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:51.680650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:51.681435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 400/792] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:51.687964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.705045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb11168-49a5-4ad5-8cc7-fa50e9483073 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.750488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.751597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.752135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.752731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.753651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:51.754030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.755599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377153b-098e-451b-8cf6-241bebcc39d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.776696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 393/793] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:51.778052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd015c-b5c8-4d30-aab0-be68f7c08c86 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.783338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd015c-b5c8-4d30-aab0-be68f7c08c86 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.783818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd015c-b5c8-4d30-aab0-be68f7c08c86 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.789088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 394/794] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:51.789088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd015c-b5c8-4d30-aab0-be68f7c08c86 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:51.834238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae49c8-0af2-4890-ae61-8028922fdc4f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.859180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.876630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.878342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.878841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.879323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.880842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:51.881408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.882953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051a0ad-ee16-4273-8c9e-f676d08ef0ec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.901568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 401/795] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:51.902941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.903327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.903434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd015c-b5c8-4d30-aab0-be68f7c08c86 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:51.903696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.921983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:51.923518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:51.938883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:51.950583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:51.952262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:51.953027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:51.953500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:51.967120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:51.969373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:51.995786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:51.996736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:51.997626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:51.997626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:51.997626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:51.997626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:52.000315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 395/796] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.001235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.011241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 402/797] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:52.011241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.132603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af7f6-e220-43f5-9467-7628bd0f5a3c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.144544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89974109-34ed-4654-b52a-1440b0336235 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.168638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.169275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.169486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.169773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.170052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.171520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:52.171747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.173175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.192750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 403/798] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:52.193478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.195326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.196670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.198045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.198570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.204925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.209778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.212281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:52.229811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.230850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 396/799] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.231859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.308800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44933-7335-44a7-ae5a-f7924d32cc01 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.344384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed03c18-9da7-45e4-86d8-ce5e40c38a9a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.344708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.345441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.345803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.345996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.356174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ada2a16-6b0f-4220-a5cb-6543b8b29821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:52.364081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.365030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:52.365890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:52.370076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 404/800] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.371394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2a25-a172-4af4-9ae8-0146ab4cd30d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.371683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2a25-a172-4af4-9ae8-0146ab4cd30d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.371966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2a25-a172-4af4-9ae8-0146ab4cd30d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.378193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 405/801] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:52.378193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e2a25-a172-4af4-9ae8-0146ab4cd30d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.385911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.386716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.387060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.387369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.394552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.396043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:52.410653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.411587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:52.412557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:52.416716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 397/802] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.417373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.450220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.458040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.489965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e2a25-a172-4af4-9ae8-0146ab4cd30d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.522872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b14f1a-bf1c-4c56-acb1-dc035ad58c5d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.526420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.527095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.527408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.527686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.528530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:52.528843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.530801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44429858-e132-4473-ad4a-b0d160c7081d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.549084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 406/803] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:52.550127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.550411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.550686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.563154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.563943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.564231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.564512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.565271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:52.565583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.566946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.567110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:52.568313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:52.579755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 398/804] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:52.579755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.643785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 407/805] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:52.644629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.691138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.694906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9c727-0886-41e1-ac99-bc2eb9777088 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.708477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.729609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.730278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.730565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.730844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.731660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:52.731977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.734017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.745806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 399/806] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:52.746585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.751817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca4943-a6b5-4c41-9838-1e0d91523c11 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.785513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.786301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.786582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.786858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.797823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.799267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:52.816184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.817091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.818000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:52.818000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:52.818000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:52.818000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:52.818000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 408/807] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.818552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.844096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4268-26c2-4c39-98ae-cf8725385169 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.877208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.877995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.878281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.878563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.888459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.890416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-005680e2-9f18-44f2-b83a-9f9efa45bbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:52.907896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.908963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.909909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:52.909909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:52.909909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:52.909909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:52.909909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 400/808] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.910368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273209be-fbb8-4881-9709-95017fa2ec8d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.910657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273209be-fbb8-4881-9709-95017fa2ec8d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.910936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273209be-fbb8-4881-9709-95017fa2ec8d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.915257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 401/809] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:52.915257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273209be-fbb8-4881-9709-95017fa2ec8d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:52.926628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0025-8047-43d5-88f7-cf1d6ec38022 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:52.928212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.937232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:52.962321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:52.963055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:52.963370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:52.963646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] PATH_INFO: `/v3/services/network` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:52.964295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:52.965656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:52.977729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:52.978737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:52.979847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 409/810] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:52.981071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.016717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273209be-fbb8-4881-9709-95017fa2ec8d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.050731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.051661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.051947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.052230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/44f05b05c1c94c44aa499f224ac20648/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.054195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=44f05b05c1c94c44aa499f224ac20648, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.055675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.068123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 402/811] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:52 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/44f05b05c1c94c44aa499f224ac20648/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:53.068907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.097494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e34935-7f90-4e78-a8d7-f7c605ae5d3f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.136494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.137164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.137459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.137742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.138495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.138884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.140980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790fc-017f-40e7-81cf-5e20f877919a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.148212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.156038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 410/812] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.157373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.157669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.157950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.176231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:53.177472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:53.189534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0a261-9b74-4eaa-859b-a0ab79175f9e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.207061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.232528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.233163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.233448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.233731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.234574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.234888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.236405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.254272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 403/813] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.254962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.257713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 411/814] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:53.258624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.367596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaaff0-ac0a-4046-8ae9-a71848e7d788 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.370381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581866a5-5c69-4906-a502-151db6cf2bd5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.399093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.405132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.405960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.406252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.406530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.407263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] RBAC: Adding query filter params (name=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.407668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.408076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d4bde-9605-49de-9107-448ac471345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.408883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d4bde-9605-49de-9107-448ac471345c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.409055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.409205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d4bde-9605-49de-9107-448ac471345c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.409450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d4bde-9605-49de-9107-448ac471345c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.418512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 404/815] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.418512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.419009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d4bde-9605-49de-9107-448ac471345c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.419009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d4bde-9605-49de-9107-448ac471345c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-548d4bde-9605-49de-9107-448ac471345c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:53.432682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:53.433597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:53.434483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 412/816] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:53.434976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d4bde-9605-49de-9107-448ac471345c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.462258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.526934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2e901-8dd4-4726-b592-d89091a9638e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.550116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d4bde-9605-49de-9107-448ac471345c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.562066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.562872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.563161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.563475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.571204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.572687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.585421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.586235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.586523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.586811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.587500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:53.587698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:53.588603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:12:53.594384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:53.594914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.594988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 405/817] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:53.594988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:53.606915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:53.608390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 15:12:53.609288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:53.611378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 413/818] 10.222.0.46 () {56 vars in 1129 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:53.612651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.654595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.704140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8d6f5-a509-49bd-ab31-7420d12ba25e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.705869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.722239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501217d-c068-4083-ad3b-f125ea86b205 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.740365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.741038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.741322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.741600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.742317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] RBAC: Adding query filter params (type=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.742630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.744175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.749988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 406/819] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.750612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.763163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.763875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.764221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.764510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.765323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.765641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.767078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.779613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 414/820] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.780377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.872940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfdf56-481d-443c-b210-4fa6faa3259c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.883170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d45e23-9882-45d0-b340-14c61c447db5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:53.895591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.911886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.912745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.913309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.913745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.914812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.915304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.916873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.918652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:53.919604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:53.919893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:53.920169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:53.921396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:53.921708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:53.923130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:53.927933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 415/821] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.928684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:53.942326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:53.944490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 407/822] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:12:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:53.945802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.037998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c76847-de3f-440e-aa4e-fb3ae307d52d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.050397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b656a-6d7f-4c28-9c85-cc03bcd49a81 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.074686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.075597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.076022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.076445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.077396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.078913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.085541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.086438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.086732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.087008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/cbf595eb1a0c44afb4d4f96413763631/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.100632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 416/823] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:12:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:12:54.101806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.114441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=cbf595eb1a0c44afb4d4f96413763631, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.116258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.136465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 408/824] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:12:53 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/cbf595eb1a0c44afb4d4f96413763631/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:12:54.141125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.186297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:54.199906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:54.258914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406455d-9316-47dc-866e-e47ad5387a4e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.305485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.306096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.306330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fce59d-650d-4a9f-8693-4d2fe0812967 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.306438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.306658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.307432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:54.307749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.309409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.316634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 417/825] 10.222.0.46 () {56 vars in 1124 bytes} [Wed Jul 6 15:12:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:54.322084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.381340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.382392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.382923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.383453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.458234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96731d-144e-4d34-9865-ca45a491db51 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.476044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.478782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.490142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:54.502502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.503506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.503803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.504088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.504858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.506228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.515278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:54.535651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 418/826] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:12:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:12:54.610220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 409/827] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:12:54.615914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.734071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849c3ef-7517-4483-8783-a97487b253bd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.772712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.773901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.774395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.774876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.786118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.787891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:54.807663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:54.808608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:54.809505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:54.809505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:54.809505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:54.809505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:54.812261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 419/828] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:54.817910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:54.927402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c8cfc-5fac-43cd-b7b0-6d6101de0383 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:54.956203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:54.965210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:54.965885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:54.966183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:54.966458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:54.967209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:54.967592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:54.968979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5ba3-758e-48d1-8df8-148a73bc10cc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:54.983374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 410/829] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:54.986642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.019611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:55.096664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44031abc-5988-4c5e-b19c-26ac7736e4ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.136839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.137643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.137933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.138215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.145265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:55.146793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:55.155168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafe9f2-6105-4cd1-952d-413b8fa5763f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.155499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafe9f2-6105-4cd1-952d-413b8fa5763f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.155781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafe9f2-6105-4cd1-952d-413b8fa5763f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.157375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 411/830] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:55.161468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:55.162453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:55.163773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 420/831] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:55.165750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.166042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.166322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.169114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.189897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:55.191888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:55.267603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 412/832] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:55.284548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:55.296397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.297482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d77868-1f45-40de-823f-93da725ddaf5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.339118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.340188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.340917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.341414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.343958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:55.344543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:55.346189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:55.363151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 421/833] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:55.368216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.416163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a60ce2-2151-4078-8c82-74af3535dd87 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.437487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:55.457569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.458399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.458685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.458962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.470176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:55.477887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:55.479308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:12:55.480196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:55.484087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 413/834] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:55.484906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.510200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492eb7f-89ba-4ac5-be5f-f355ed12c8f9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.556059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.558161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.558666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.559173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.560269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:55.560796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:55.563553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:55.572515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 422/835] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:55.593835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8a187-5af6-4047-81c1-b0119cb92eb5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.633460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.634164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.634453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.634737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.635549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:55.635864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:55.637282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:55.648074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 414/836] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:55.653712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.675968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:55.776916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe446-455e-4f10-8e1b-c249dc23054e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:55.813086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:55.813927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:55.814180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:55.814461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:55.825529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:55.827585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:55.845120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:55.846070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:55.846978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:55.846978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:55.846978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:55.846978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:55.846978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 423/837] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:55.851909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:55.898925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:55.964013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec09c80-cefc-4d7c-a776-4320a8600383 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.001946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.002625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.002915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.003197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.003954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:56.004265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:56.005660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:56.019257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 415/838] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:56.024738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.095051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:56.134204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edc2f8-117f-42aa-a770-84cfa48e5bfa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.172009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.173148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.173617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.174076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.182136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:56.184008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:56.199272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:56.200283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 15:12:56.201257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:56.203590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 424/839] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:56.215356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.319828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:56.323063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c6b6-f98d-46d6-b9f8-b1bad2d6ed50 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.360688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.361296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.361584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.361865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.362714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:56.363028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:56.364509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691f540-d364-4720-a522-49d62d30bd46 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:56.382588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 416/840] 10.222.0.46 () {56 vars in 1135 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:56.388154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.447040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b2bf-315a-4798-a13c-b93621c3b3f0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.447380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b2bf-315a-4798-a13c-b93621c3b3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.447633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235b2bf-315a-4798-a13c-b93621c3b3f0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.449162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 417/841] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:56.454325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.454861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.455384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.475415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:56.477398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:56.498779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c1d98-71c9-4bf3-b741-22a04acd215d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.524834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:56.534461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.535130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.535496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.535782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.569188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 418/842] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:12:56.591210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.618854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:56.620314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec6eb17-3097-452e-9be3-286a1e2763bc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:56.692876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:56.707502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984bc1a0-90c8-44a9-86d6-45ee39b95205 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.729422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 425/843] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:12:56.730880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007053a2-6992-4c67-bf51-d6ccceb1428f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.731175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007053a2-6992-4c67-bf51-d6ccceb1428f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.731476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007053a2-6992-4c67-bf51-d6ccceb1428f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.732881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 426/844] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:56.734803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007053a2-6992-4c67-bf51-d6ccceb1428f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.751233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.752457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.753051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.753528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.761346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:56.768631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b5c88f02-ac23-4b9a-925d-22e3bb40afd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:12:56.778450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:56.779416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:12:56.780829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:56.781469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 419/845] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:56.782319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.782511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.782775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.799506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:56.800704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:56.847866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007053a2-6992-4c67-bf51-d6ccceb1428f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:56.871782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 420/846] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:56.883431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:56.884351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:56.884470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.884807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:56.885179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:56.896904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:56.898355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:56.917018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:56.918001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:56.919174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:56.919174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:56.919174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:56.919174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:56.919174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 427/847] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:56.919494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:56.980032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.001345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e5c1f2-95d3-4338-ab34-d9691abe55ff None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.003097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.027746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc3388-9785-4187-8154-f6d3b9792edb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.034487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.035162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.035471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.035745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.036498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.036812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.038220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.046531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 421/848] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.047399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.069574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.070682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.071116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.071560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.072476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.073964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:57.086858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:57.087889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.088768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 428/849] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:57.089600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.152682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dbe0d-48fd-44b2-ba37-21d7000caee9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.185740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.186938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.187308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.187531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.188246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.188552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.189913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.194187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cf086-6079-444c-9b1d-0fc449c65163 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.199243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.203572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 422/850] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.204508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.237358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.238155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.238439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.238717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.252697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.254159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.255980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:57.274525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:57.275732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:57.276853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:57.276853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:57.276853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:57.276853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.280501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 429/851] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:57.281233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.321265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b996ba6-0da8-4bb4-be20-f6ab6f7cf59f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.356818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.357493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.358315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.358599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.359352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.359674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.361047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.365919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 423/852] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.366533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.390613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b9434-adb1-4f91-83b5-2824a0f3e061 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.426499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.427436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.427807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.428216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.434196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.435541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.438003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:57.452677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:57.453651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.454567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 430/853] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:57.455059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.484795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f94bf-cbce-418e-abbb-abac0fd55416 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.515041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.519806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.520480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.520771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.521053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.521781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.522093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.523492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.534880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 424/854] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.535721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.566723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5143b-8f29-4259-a1b2-15b190a31ca2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.605792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.606570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.606930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.607211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.608455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.608765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.610157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.614692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 431/855] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.615559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.659494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1878f-c2e0-4751-984c-bdbc68a0d13c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.698001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.714874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.716210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.716755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.718198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.719246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.719757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.722095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.735356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0875c6a-3b1f-4ff0-a685-00fb1b596911 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.742026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 425/856] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:57.748735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.761306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:57.782906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.784045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.784484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.784928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.795106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.797150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3f226fbc-4fec-4583-889e-25cda956c8cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:12:57.816649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:57.817635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:12:57.818643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 432/857] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:57.820550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154293-c26f-40c8-a85e-ad75e4c5bb13 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.820844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154293-c26f-40c8-a85e-ad75e4c5bb13 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.821293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83154293-c26f-40c8-a85e-ad75e4c5bb13 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.823395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 433/858] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:57.827791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83154293-c26f-40c8-a85e-ad75e4c5bb13 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:57.870204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3dba6-10be-4f6b-82e4-540ad7eeae01 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:57.905920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:57.906589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.906877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.907163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.908039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=44f05b05c1c94c44aa499f224ac20648) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:57.908358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:57.909796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c9bda-f611-49c1-a2e4-110eb84de52b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:57.915831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 426/859] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/role_assignments?user.id=44f05b05c1c94c44aa499f224ac20648&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:12:57.917026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:57.917318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:57.917598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:57.944785 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:57.946783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:57.951841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83154293-c26f-40c8-a85e-ad75e4c5bb13 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.002314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.003383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.003893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.004424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.005503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:58.006021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.007747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.012965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.038636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 434/860] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:12:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:58.044610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.055912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 427/861] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:58.069970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.088503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.174949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402f926-d6cc-4a5d-ad9e-db9af3e39423 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.194084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834ad1e-94ae-4aa1-9a0d-d95076de566a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.215610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.216493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.216932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.217926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.236545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.237643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.238023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.241330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.241330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.241330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:58.255391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:58.256570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:12:58.257565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:58.258399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 428/862] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:58.265264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.310231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.311833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.316500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.355613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.376935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fc204-3a3f-4ab7-b2dc-db7e408dd98e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.416037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.416869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.417157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.417443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.418482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:58.419064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.420778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.426368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 429/864] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:58.426618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 435/864] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 15:12:58.431400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.432074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.548808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417da3bd-f123-47a3-a00f-b82042c1cbbe None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.559092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.562117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135895fc-a381-4314-b203-0ff85c5f849a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.596471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.597592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.598314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.598774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.606220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.607422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.607739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.608027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.609248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:58.609565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.610566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.611201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.612141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.613640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:58.617691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 436/865] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:12:58.635836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:58.636816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:58.637727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:12:58.637727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:12:58.637727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:12:58.637727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:58.642481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 430/866] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:12:58.643996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.761115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f753-5fab-4253-ae43-bc35b79edf45 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.798433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.799161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.799411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.799687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.800419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:58.800726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.802097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.819331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 437/867] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:58.821136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:58.924982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec66af-ea91-43c4-a78a-05e53c966e29 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:58.960376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:58.961173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:58.961452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:58.961797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:58.968764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:58.970207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:12:58.984668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:12:58.986658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:12:58.988779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:12:58.992367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 431/868] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:12:58.992567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:59.056011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:59.099918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eae65-3353-4f11-9e4f-da13545c3972 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:59.138526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:59.139654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:59.140151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:59.140658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:59.141661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:59.142193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:59.143897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2bc6-6368-4505-8ab6-74293b68a7af None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:59.157641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 438/869] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:59.163586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:59.185351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:59.270688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8edf8-2827-44d8-8ef2-a3cc511249a1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:59.306290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:59.306872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:59.307154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:59.307457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:59.308298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=cbf595eb1a0c44afb4d4f96413763631) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:12:59.308612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:59.310038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c43ff4-a177-429f-ac69-67822032bd72 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:59.323318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 432/870] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:12:59 2022] GET /identity/v3/role_assignments?user.id=cbf595eb1a0c44afb4d4f96413763631&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:12:59.696774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9059-e6c8-411d-abb7-bc0893db18b8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:59.697261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9059-e6c8-411d-abb7-bc0893db18b8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:59.697261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b9059-e6c8-411d-abb7-bc0893db18b8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:59.698916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 439/871] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:12:59.704629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235d62-e3f1-44f0-b6b3-9a9dbdc7ba5e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:59.704930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235d62-e3f1-44f0-b6b3-9a9dbdc7ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:59.705212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235d62-e3f1-44f0-b6b3-9a9dbdc7ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:59.722883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-76235d62-e3f1-44f0-b6b3-9a9dbdc7ba5e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:12:59.724125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-76235d62-e3f1-44f0-b6b3-9a9dbdc7ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:12:59.793357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 433/872] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:12:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:12:59.816397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b9059-e6c8-411d-abb7-bc0893db18b8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:12:59.921127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b9059-e6c8-411d-abb7-bc0893db18b8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:12:59.942579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:12:59.957449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:12:59.958127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:12:59.958416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:12:59.958751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:12:59.959519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:12:59.961001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab6f04-33f2-4060-85b1-af2edde489ec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:12:59.977431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 440/873] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:12:59 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:00.094404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9c730-54b9-42e3-afd8-d9d6845dc329 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.094919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9c730-54b9-42e3-afd8-d9d6845dc329 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.094996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9c730-54b9-42e3-afd8-d9d6845dc329 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.096502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 434/874] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:00.102326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.102959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.103477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.121385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:00.122940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:00.204407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 441/875] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:00.226290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9c730-54b9-42e3-afd8-d9d6845dc329 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:00.241418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:00.336953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9c730-54b9-42e3-afd8-d9d6845dc329 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:00.374676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:00.375520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.375805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.376139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.376804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:00.378288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:00.390802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:00.391863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:00.392705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 435/876] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:00.398035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:00.504450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3220-222d-4174-a843-9b9062bcdb04 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:00.512290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:00.541446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:00.542125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.542414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.542698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.543453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:00.544024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:00.545911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75bf1e-2233-4658-8c34-479b4781a13f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:00.551129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 442/877] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:00.560285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:00.643653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:00.667882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584beb-25d6-4432-9684-8e59fbe0a68f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:00.704678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:00.705352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.705639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.705921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.706635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:00.706949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:00.708349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc625ec7-c674-4177-9361-487228fe9503 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:00.713348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 436/878] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:00.782591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b3435-34d4-4b47-8cee-5b074b680ece None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.783106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b3435-34d4-4b47-8cee-5b074b680ece None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.783106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b3435-34d4-4b47-8cee-5b074b680ece None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.784748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 443/879] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:00.790392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263531c-22a6-43c1-828d-ff67b4c5b7d7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:00.790692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263531c-22a6-43c1-828d-ff67b4c5b7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:00.790971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263531c-22a6-43c1-828d-ff67b4c5b7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:00.809864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e263531c-22a6-43c1-828d-ff67b4c5b7d7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:00.811118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e263531c-22a6-43c1-828d-ff67b4c5b7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:00.889443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 437/880] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:00.912459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b3435-34d4-4b47-8cee-5b074b680ece None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:01.022820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b3435-34d4-4b47-8cee-5b074b680ece None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:01.060564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:01.061509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:01.061949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:01.062404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:01.063321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:01.064845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e16578-ab24-4249-8186-3ec2f14b7675 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:01.082459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 444/881] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:00 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:01.530261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f7d03-57ab-4268-86b1-851fe20bdbe0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:01.530780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f7d03-57ab-4268-86b1-851fe20bdbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:01.530866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f7d03-57ab-4268-86b1-851fe20bdbe0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:01.532500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 438/882] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:01.537864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:01.538161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:01.538443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:01.555365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:01.556562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:01.581452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:01.636277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 445/883] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:01.659284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f7d03-57ab-4268-86b1-851fe20bdbe0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:01.733373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:01.767072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f7d03-57ab-4268-86b1-851fe20bdbe0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:01.802331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:01.803076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:01.803446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:01.803741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:01.814904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:01.823664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:01.824611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:01.825602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 439/884] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:01.833468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:01.941746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca6bb0-150d-4ece-b0c4-f527e5743265 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:01.977267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:01.977945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:01.978245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:01.978521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:01.979312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:01.979633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:01.981015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551ee3d-90fb-4ac4-9c6e-00b8695ce127 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:01.990147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 446/885] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:01.995857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.096694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5cae-8aa5-4586-bb0d-bfa402276c15 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.097318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5cae-8aa5-4586-bb0d-bfa402276c15 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.097805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f5cae-8aa5-4586-bb0d-bfa402276c15 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.099703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 447/886] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:13:02.104245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263fec-765d-4795-a312-d2517bd08493 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.104881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263fec-765d-4795-a312-d2517bd08493 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.105686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263fec-765d-4795-a312-d2517bd08493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.108709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867c21d-2cd8-434a-8ccc-9348e830507e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.127433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-97263fec-765d-4795-a312-d2517bd08493 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:02.128941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97263fec-765d-4795-a312-d2517bd08493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:02.138148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.145498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.146342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.146678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.146964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.151554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.159483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.160985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:02.180309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:02.181331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:02.182244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:02.182244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:02.182244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:02.182244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:02.182244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 440/887] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:02.187665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.219027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 448/888] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:02.242749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263fec-765d-4795-a312-d2517bd08493 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.298491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3701-68a3-41ab-85c9-a4c480013ceb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.334864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.335879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.336339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.336763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.337646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:02.338098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.339648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.357525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 441/889] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:02.359655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.368201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263fec-765d-4795-a312-d2517bd08493 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.406402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.408183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.409097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.411473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.411473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.411473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.411833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:02.426479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:02.427662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:02.428516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:02.431291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 449/890] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:02.433672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.449700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.491619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d300a-6f5f-4ade-8d5e-152144f4a5ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.537257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.539308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.539740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.540161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.544382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb32511-4b8f-4b3f-aa4f-795ae28a66cf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.550377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.552015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:02.571258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:02.572454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:02.573600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:02.576871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 442/891] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:02.582459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.588023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.588699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.588985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.589258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.589974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:02.590294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.591755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.603302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 450/892] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:02.604177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.703439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd12e65-85ce-4597-a74b-c596cddc884d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.704509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.706156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.735315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc11e4-e52f-4145-90ea-185b9c6c0703 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.744317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.745303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.745813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.746392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.747508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:02.748076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.749783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.769492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 443/893] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:02.776257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.783724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.784672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.785206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.785681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.786701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:02.787233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.788923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.796659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 451/894] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:02.802834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.899134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ab856-3974-4091-b2fc-0ef4d8e5affb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.916147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2105-e255-4f31-ab2f-64f114b19623 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:02.936870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.937527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.937817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.938096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.953268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:02.953992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.954228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.954515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.955346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] RBAC: Adding query filter params (interface=public, service_id=a140fcc2ce524e74bd1b211210bbc669, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:02.955675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:02.957041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20cfc1-4ad0-428e-baac-e6fddad9d30c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:02.962280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 452/895] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/endpoints?service_id=a140fcc2ce524e74bd1b211210bbc669&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:02.963306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa834-5fc1-4f60-a48c-fb5496610da8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:02.963601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa834-5fc1-4f60-a48c-fb5496610da8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:02.963884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa834-5fc1-4f60-a48c-fb5496610da8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:02.965232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 453/896] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:02.968344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa834-5fc1-4f60-a48c-fb5496610da8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:02.972141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:02.974136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.015019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.016518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910d65f-a486-4df3-be5e-82dc644f1e08 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.083664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa834-5fc1-4f60-a48c-fb5496610da8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.120761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.121439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.121738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.122005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.122692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.128847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 444/897] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:03.128984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.129063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e6eb-2f1b-4546-81e7-8a88009f7999 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.129063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e6eb-2f1b-4546-81e7-8a88009f7999 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.129063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268e6eb-2f1b-4546-81e7-8a88009f7999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.139684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 454/898] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:02 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:13:03.139684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.154758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9268e6eb-2f1b-4546-81e7-8a88009f7999 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:03.156127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9268e6eb-2f1b-4546-81e7-8a88009f7999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:03.218695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.235983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.254783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 445/899] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:03.256191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d36f5-cfbf-4a8e-9b40-9a022eab6e4f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.256483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d36f5-cfbf-4a8e-9b40-9a022eab6e4f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.256763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d36f5-cfbf-4a8e-9b40-9a022eab6e4f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.258207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 446/900] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:03.263581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.264439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.264723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.274767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26f04-2727-422c-8f09-30323f345593 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.282331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:03.283613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:03.315379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.315880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.316167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.316446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.328874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.331052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:03.349920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:03.350895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:03.351904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:03.351904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:03.351904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:03.351904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:03.351904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 455/901] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:03.352191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.363253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 447/902] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:03.364092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.457198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bb74-48f3-44b0-b7d3-bfd5e847f413 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.478778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.486828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2e12e-d7f7-4833-8a99-76344a0c4e11 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.493164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.493837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.494121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.494405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.495129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.496540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.512946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.519612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 456/903] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:03 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:03.520436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.542373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.544297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.544587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.544867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.546158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:03.546467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.547926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.562051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 448/904] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:03.567640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.668167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da0e42-1173-41e1-9344-0496fdf6c54a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.690767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73966041-2caf-427c-9dfb-2c33d680f740 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.712003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.712824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.713110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.713386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.714048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.715433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:03.729395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:03.730363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:03.731259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 457/905] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:03.735182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.736332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.736796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.737122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.737793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.745073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.746465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:03.761815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:03.762749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:03.763672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 449/906] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:03.769533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.793973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.821140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:03.860160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a950d4-5a19-4bf2-a099-ae9bef2f82ca None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.897557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.898293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.898587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.898865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.899617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:03.899961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.901546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.907841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 458/907] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:03.908830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824b5be-6955-4635-9ae8-336dd3986261 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:03.912912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:03.956402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:03.957078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:03.957364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:03.957646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:03.958448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:03.958760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:03.960620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:03.979365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 450/908] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:03.981426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:04.040831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c2f3d-fcfb-446c-9198-39b1d29a8946 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:04.059298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:04.088509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:04.089284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:04.089574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:04.089850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:04.090561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:04.090869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:04.092575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:04.094198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:04.098315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe0c2-1256-451e-b1a5-fb7c289d0dda None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:04.104867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 459/909] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:04.105944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:04.133528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:04.134199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:04.134494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:04.134959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:04.135917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] RBAC: Adding query filter params (role.id=e093dc275c994361b21cb90aceb44ff2, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=cbf595eb1a0c44afb4d4f96413763631) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:04.136538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:04.137814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:04.150206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 451/910] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:03 2022] GET /identity/v3/role_assignments?user.id=cbf595eb1a0c44afb4d4f96413763631&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=e093dc275c994361b21cb90aceb44ff2 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:04.226427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a40ca-5601-4580-8539-42e97d1b5872 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:04.264928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:04.265614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:04.265900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:04.266179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:04.267010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] RBAC: Adding query filter params (interface=public, service_id=c622aac5193442639692ca4eee8fca84, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:04.267362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:04.268787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f78f9-718d-4012-897c-4a91b1524bba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:04.275672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 460/911] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:13:04 2022] GET /identity/v3/endpoints?service_id=c622aac5193442639692ca4eee8fca84&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:04.281998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:04.388005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9980f-2116-4348-ac11-ad9213ac9baa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:04.424556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:04.425231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:04.425517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:04.425793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:04.426482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:04.427882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496e1f6-f766-4eea-afdb-4e7cd244ac5d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:04.434123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 452/912] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:04 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:13:05.198979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fa7-3ca9-432f-922f-1ed9a49d3843 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.199556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fa7-3ca9-432f-922f-1ed9a49d3843 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.199556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8fa7-3ca9-432f-922f-1ed9a49d3843 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.201809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 461/913] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:05.207011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.207458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.207632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.224231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:05.225419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:05.259614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:05.315330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 453/914] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:05.332103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8fa7-3ca9-432f-922f-1ed9a49d3843 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:05.371169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:05.439199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8fa7-3ca9-432f-922f-1ed9a49d3843 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:05.473475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:05.474275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.474563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.474845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.475533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:05.476892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:05.489290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:05.490273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 15:13:05.491141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.495322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 462/915] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:05.496996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:05.602111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da34dd-06c0-4e8a-a847-f5a6d58ae123 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:05.641370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:05.642047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.642332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.642607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.643344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:05.643658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:05.645040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f2f7c-d20f-4083-b53c-98f8a2254f07 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:05.653911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 454/916] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:05.656526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:05.692283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485b22-e750-4d55-9ab5-31f625a3f20b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.692620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485b22-e750-4d55-9ab5-31f625a3f20b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.692839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485b22-e750-4d55-9ab5-31f625a3f20b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.694571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 455/917] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:05.699976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.700365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.700737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.718980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:05.720376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:05.735719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:05.765803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:05.769095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103511-4964-4290-b3d1-7332775013c0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:05.804012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 456/918] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:05.807280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:05.807997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.808332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.808621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.809342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:05.809655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:05.811055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:05.820647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 463/919] 10.222.0.46 () {56 vars in 1114 bytes} [Wed Jul 6 15:13:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:05.821962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:05.824159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:05.930941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d0ec6-0718-46b1-ae4e-09e2170e2b34 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:05.931441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eead0-217a-46ed-bdab-7054d2726312 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:05.968258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:05.968681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:05.968926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.969214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.969493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.970254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:05.970598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:05.971085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:05.971580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:05.971806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:05.972812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:05.974375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:05.988234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:05.989402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:05.990363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:05.998866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 464/920] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:05.999053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 457/921] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:06.001662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.039602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.122833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdb2-85c9-4f3e-bb36-45ae340b14bd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.167667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.168979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.169267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.169548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.170305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:06.170616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.172166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c4f61-99ba-4ff5-9d3c-313bdbfe36c1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.177488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 465/922] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:06.184063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.189793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.192531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba151f-2e3c-4e44-8c00-34050a4b0ea5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.192811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba151f-2e3c-4e44-8c00-34050a4b0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.193134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba151f-2e3c-4e44-8c00-34050a4b0ea5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.194829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 466/923] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:06.203102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.203741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.203741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.224549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:06.225995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:06.311310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 467/924] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:06.327061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f5799-2917-47a5-916f-2129e3896711 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.334695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.364622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.365685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.366196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.366688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.367584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.367845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:06.368363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.369941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.381326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 458/925] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:06.383039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.457689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.463009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3d2b-fdf6-4ddd-85b6-53b49334f5e2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.493969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503eadf-2838-47f5-9fb9-9c02f0fcca78 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.507792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.508570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.508854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.509132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.523461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:06.533203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:06.533627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:06.534567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:06.540310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 468/926] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:06.542171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.542341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.542904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.543399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.544653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] RBAC: Adding query filter params (interface=public, service_id=9e7a5a8e870f40f9a62f78f62c1f54af, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:06.545196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.546770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.559325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 459/927] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/endpoints?service_id=9e7a5a8e870f40f9a62f78f62c1f54af&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:06.561480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.632581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.657593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a002f-beb8-4a22-ae6b-d83d34aea359 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.670675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5181e-76d7-4acf-9378-03b706836ba9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.698399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.699075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.699398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.699680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.700437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:06.700750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.702014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.702681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7de1f-d81c-4c1f-a2c5-451ee1227e91 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.708077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.708754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.709044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.709331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.710050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.711490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fc479-a223-468a-b79d-a63cdbdccd9a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.712439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 469/928] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:06.714010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8baaa-c810-4206-b6c2-5d9a35f1b506 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.714363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8baaa-c810-4206-b6c2-5d9a35f1b506 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.714703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8baaa-c810-4206-b6c2-5d9a35f1b506 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.716179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 470/929] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:06.720754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8baaa-c810-4206-b6c2-5d9a35f1b506 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.722456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 460/930] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:13:06.723708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.724006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.724290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.742872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:06.744436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:06.839257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 461/931] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:06.862647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8baaa-c810-4206-b6c2-5d9a35f1b506 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:06.863099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.907322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:06.908602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:06.908602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:06.908812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:06.917975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:06.921366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:06.923383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:06.948801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:06.949961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:06.951013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:06.951013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:06.951013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:06.951013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:06.952932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 471/932] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:06.958603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:06.987846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b05b-5d5f-4361-aa49-ebe828d10565 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.023586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:07.028260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.029562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.030298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.030685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.031744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.033800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:07.054355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:07.055402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:07.056276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 462/933] 10.222.0.46 () {56 vars in 1132 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:07.062096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.080661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2df27a-9c78-4944-9d21-97fd62da0aba None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.119584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.120184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.120472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.120806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.121658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:07.121974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.123524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.143258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 472/934] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:07.143258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.180230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe4277-e22e-4c77-9c99-5fc38c39b22a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.214211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:07.219253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.220361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.220748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.221119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.221961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:07.222376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.224159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.230269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 463/935] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:07.244627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.280229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77095e-cf31-49b8-8e3b-c2dbfed17fde None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.318363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:07.323765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.324591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.324877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.325157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.334024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.335493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:07.350819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:07.352048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:07.353027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:07.355543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 473/936] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:07.358382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.375540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3e023-4493-4088-bd56-9986de78d157 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.410975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.412087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.412585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.413057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.413972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:07.414486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.416277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.423115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 464/937] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:07.428894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.477574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda50059-fa42-431f-8f5b-c6e2f77da210 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.494980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:07.524770 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.525815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.526308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.527330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.528387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:07.529982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.532321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.556452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 474/938] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:07.556803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30dcd-407a-417f-9c73-ca8839b8155a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.562751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.578294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:07.597657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.598625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.599107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.599645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.600599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.602225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.631444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 465/939] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:07.677124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b075f53-06d1-4b59-b5b5-601bee51eb3b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.725212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.726074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.726369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.726652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.738712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.740310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:07.760177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:07.761163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:07.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:07.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:07.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:07.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:07.762061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 475/940] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:07.768131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.878432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db346-8518-4bfa-bd41-c4b195ee9773 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:07.915765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:07.916417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:07.916708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:07.917034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:07.917732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:07.918048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:07.919721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:07.932777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 466/941] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:07.939340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:07.960038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.046482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c29d2-68eb-4506-87a2-9b23917f58e6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.082968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.083855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.084148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.084432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.091545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.093024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f7670e28-d145-458d-b844-4eab0704aa18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:08.106820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:08.107787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.108688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 476/942] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:08.114051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:08.149908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.156991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca7443-75d8-4e86-adc1-bd5dba42b29f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.157288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca7443-75d8-4e86-adc1-bd5dba42b29f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.157568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca7443-75d8-4e86-adc1-bd5dba42b29f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.159069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 477/943] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:08.164881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.165190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.165476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.183052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:08.184245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:08.236984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef897fcc-8fee-4671-847f-9ab19303ae00 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.271256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 478/944] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:08.280118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.280797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.281089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.281371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.282181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:08.282501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.283976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.292091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:08.298423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 467/945] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:08.304792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:08.311602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.405063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.414231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f30c6-4c55-40f2-846e-c0b6ba6260f0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.423269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ea855-fca8-4bc5-97a2-3a530339ee91 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.451713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.453016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.453500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.453989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/cbf595eb1a0c44afb4d4f96413763631/roles/e093dc275c994361b21cb90aceb44ff2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.456487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=cbf595eb1a0c44afb4d4f96413763631, role_id=e093dc275c994361b21cb90aceb44ff2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.458135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.465360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.466009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.466404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.466588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.467380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.468821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.470253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING keystone.server.flask.common [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 06 15:13:08.475739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 468/946] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:13:08 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/cbf595eb1a0c44afb4d4f96413763631/roles/e093dc275c994361b21cb90aceb44ff2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:13:08.495515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 15:13:08.495515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 15:13:08.495515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [parameters: {'id': '465e1dd2c2ec4c7c994d628ddb5ae7d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 15:13:08.495515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 465e1dd2c2ec4c7c994d628ddb5ae7d9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 465e1dd2c2ec4c7c994d628ddb5ae7d9. Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:13:08.505551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 06 15:13:08.506472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 06 15:13:08.507518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 15:13:08.508427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 15:13:08.509353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:13:08.510162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'id': '465e1dd2c2ec4c7c994d628ddb5ae7d9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.511063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:08.519198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 465e1dd2c2ec4c7c994d628ddb5ae7d9. Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:08.521398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 479/947] 10.222.0.46 () {60 vars in 1140 bytes} [Wed Jul 6 15:13:08 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:08.524844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:08.640294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd50036-a4c7-4831-9a8b-0aadc34b29e7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.677035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.678178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.678685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.679186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.686826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.688541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f9fbda96-6170-4fbe-bb65-9438d43f9429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:08.703654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:08.704589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:08.705496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:08.711305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 469/948] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:08.714375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:08.782553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.825899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4b2df-3ba2-422b-bd5f-55744b7df5ee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:08.856180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ffc6f-bf0e-459f-943f-f5c744cdf9a8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.856518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ffc6f-bf0e-459f-943f-f5c744cdf9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.856756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ffc6f-bf0e-459f-943f-f5c744cdf9a8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.858960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 470/949] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:08.864038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.864281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.864539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.869431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:08.870111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:08.870401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:08.870682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:08.871492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:08.871805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:08.873505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:08.882671 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:08.884295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:08.894231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:08.894361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 480/950] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:08.966036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 471/951] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:08.989902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:09.101815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d1636-8fdf-40dd-b028-706e15fe41a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:09.125260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:09.136911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:09.137726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.138058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.138339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.139555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:09.140975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:09.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:09.154931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:09.155822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 481/952] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:09.163174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:09.272171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cbef0-2e52-4298-a66e-44eea3ff1c50 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:09.308217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:09.308896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.309190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.309471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.310197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:09.310556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:09.312020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:09.317520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 472/953] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:09.324301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:09.429603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67153-8cd2-48cc-a914-1a43507f8e9f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:09.466838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:09.467548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.467839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.468123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.468843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:09.469156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:09.470526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206782b-846a-4ee3-a688-c682f9f8752d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:09.475961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 482/954] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:09.481561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:09.493375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:09.593116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7fa00-ae81-49a8-8291-256f855ff7fc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:09.628890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:09.629571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.630197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.630479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.631263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:09.632643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf778935-1e39-4df0-b597-4c0f4d89a5ca None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:09.656276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 473/955] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:09.765324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04678411-b7d3-48fa-82b1-0167b506ee4c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.765690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04678411-b7d3-48fa-82b1-0167b506ee4c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.765890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04678411-b7d3-48fa-82b1-0167b506ee4c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.767568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 483/956] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:09.773046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:09.773325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:09.773606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:09.791090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:09.792325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:09.855602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:09.871790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 474/957] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:09.894575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04678411-b7d3-48fa-82b1-0167b506ee4c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:09.950456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:10.001048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04678411-b7d3-48fa-82b1-0167b506ee4c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:10.034803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:10.035636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.035920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.036196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] PATH_INFO: `/v3/services/compute` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.036849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:10.038346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:10.051237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:10.052333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:10.053239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 484/958] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:10.058630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:10.167556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606fa40-3efe-4e9e-964f-7b2fcab0b30c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:10.204031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:10.205085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.205597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.206079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.207003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:10.207643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:10.209214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bc66-53cb-4960-844c-f37a32388921 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:10.214809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 475/959] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:10.220742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:10.331952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e619035-1a15-421f-8d99-1fa2551bbbba None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:10.348961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:10.367647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:10.368314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.368599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.368877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.369587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:10.369896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:10.371391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87787d-fc21-4aad-9637-f4b108b4aa20 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:10.376386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 485/960] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:10.658955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73493-ade4-4e36-82cb-32468e70df1f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.659523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73493-ade4-4e36-82cb-32468e70df1f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.659605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b73493-ade4-4e36-82cb-32468e70df1f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.661235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 476/961] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:10.666381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.666673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.666956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.684479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:10.686055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:10.759330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 486/962] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:10.778828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73493-ade4-4e36-82cb-32468e70df1f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:10.839902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:10.888074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b73493-ade4-4e36-82cb-32468e70df1f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:10.927466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:10.928221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:10.928801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:10.929089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:10.941440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:10.950436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:10.952055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:10.952902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 477/963] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:10.958372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.062214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a4ade-5dff-403a-a991-758cd199a3a4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.073496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.097593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.098270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.098555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.098833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.099616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:11.099923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.101283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ce561-90c5-4034-9e32-c8f5a72d8ef6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.110225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 487/964] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:11.116551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.151654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624745-eb3e-43e8-bf8b-cb3a71fcf9a2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.152101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624745-eb3e-43e8-bf8b-cb3a71fcf9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.152482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe624745-eb3e-43e8-bf8b-cb3a71fcf9a2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.154280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 488/965] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:13:11.159084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.159474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.159819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.176759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:11.178171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:11.234980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96114f-ed48-42d1-844b-23647305a3f5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.263356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 489/966] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:11.267728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.288916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.302380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.303691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.304143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.304860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.320679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.323260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:11.346256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:11.347379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:11.348531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:11.348531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:11.348531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:11.348531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:11.351145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 478/967] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:11.356031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.397050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8d8a7-2dc8-4707-b862-283a9d94effa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.398914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.434998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.436213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.436602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.437120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.437808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.439480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.455305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 15:13:11.455305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 15:13:11.455305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [parameters: {'user_id': '85d4785f1c024ab8be3aab681990e2fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 15:13:11.455305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:13:11.465125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:13:11.466083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 15:13:11.467124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 15:13:11.468115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 15:13:11.469013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 15:13:11.469880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'user_id': '85d4785f1c024ab8be3aab681990e2fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:11.470807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 06 15:13:11.473214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 490/968] 10.222.0.46 () {60 vars in 1131 bytes} [Wed Jul 6 15:13:11 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:11.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.502435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4a081-4462-4412-95fa-bc10b355568c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.541890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.542598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.542890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.543174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.544302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:11.544625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.546593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.565852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 479/969] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:11.567410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.597351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.612444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b36193-d97f-4bb3-8c68-82ec105d63af None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.650335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.651171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.651497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.651777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.660014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.661654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:11.679232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:11.680398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 15:13:11.681414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:11.685226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 491/970] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:11.686344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.692394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4045-2421-4ece-8474-a3f662b4506d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.712292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.735922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.737152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.737588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.738006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.745974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.747923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5b85b1c0-f5d3-4647-a49f-8a069ff59d5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:11.763970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:11.765106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 15:13:11.766166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:11.771618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 480/971] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:11.771618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47731a8-8d8c-4c25-a9af-de020bf6acca None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.771618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47731a8-8d8c-4c25-a9af-de020bf6acca None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.771618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47731a8-8d8c-4c25-a9af-de020bf6acca None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.771618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 481/972] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:11.775122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47731a8-8d8c-4c25-a9af-de020bf6acca None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:11.793333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747cc7c-55b4-49ca-8259-15fb620c7c69 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.829439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.830105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.830390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.830667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.831501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:11.831811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.833213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f40ee-02ec-4a06-972b-05a58842ff13 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.866314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:11.879260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 492/973] 10.222.0.46 () {56 vars in 1101 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:11.879260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.879260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.879260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.892200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47731a8-8d8c-4c25-a9af-de020bf6acca None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:11.899453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:11.901781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:11.940641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:11.941358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:11.941714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:11.942144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:11.943189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:11.943616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:11.945397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:11.968508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 482/974] 10.222.0.46 () {56 vars in 1149 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:11.973794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.012649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 493/975] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:12.037228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.052226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.102917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d4952-54c2-4b06-b376-962718cc84d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.150235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.151494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.151971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.152379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.158849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fd712-a975-4fb4-a1ae-a4e513c19359 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.168882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.201610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.202724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.203029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.203339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.204006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.205378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:12.219079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:12.220300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:12.221223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 494/976] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:12.226057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.230981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.232530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.312360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.343929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 483/977] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:11 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:12.344108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c5666-3d64-4ee5-b010-cdfaef918fe5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.345561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.378953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.379657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.379971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.380218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.381315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:12.381315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.382573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.387692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 495/978] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:12.393630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.396930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.458445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14b3db-18c7-4602-9912-f8ba6cc61727 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.493666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.494452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.494737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.495014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.495147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7aa4d-9b1b-4eaa-a602-08b972c1bf13 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.505981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.507459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d494f6c8-45b6-44d4-ae23-94905fe150d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:12.525585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:12.526563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:12.527491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:12.527491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:12.527491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:12.527491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:12.527491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 484/979] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:12.528196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15140-114b-4a42-922f-a7f206b730e5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.528453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15140-114b-4a42-922f-a7f206b730e5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.528670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba15140-114b-4a42-922f-a7f206b730e5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.529966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 485/980] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:12.532200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15140-114b-4a42-922f-a7f206b730e5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.534469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.535347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.535640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.536732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.537459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:12.537772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.539132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e58ae-eade-4dbf-94bf-0860eb1b8bb1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.549956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 496/981] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:12.549956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.549956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.549956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.552637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.563884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:12.565062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:12.635312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.635492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 497/982] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:12.635492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.639300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba15140-114b-4a42-922f-a7f206b730e5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.672893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.673556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.673842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.674120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.674842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:12.675150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.676559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.688016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 486/983] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:12.688749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.730662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa54d-9996-41c5-87d2-b614c6f55302 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.764347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.765003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.765294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.765572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.766374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] RBAC: Adding query filter params (interface=admin, service_id=9e7a5a8e870f40f9a62f78f62c1f54af, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:12.766693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.768066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.769518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.777566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 498/984] 10.222.0.46 () {56 vars in 1249 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/endpoints?service_id=9e7a5a8e870f40f9a62f78f62c1f54af&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:12.777566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.785636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a72935-59f7-4751-be7e-70d9ae6ce5e2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.820153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.820825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.821113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.821400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.822127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.823541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.836426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 487/985] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:12 2022] POST /identity/v3/services => generated 242 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:12.836426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.861052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:12.884539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c089ef-1454-4a1d-a514-9e4341f05d7a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:12.930329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:12.931281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:12.931625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:12.931868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:12.943791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:12.945414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:12.961299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:12.962322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:12.963284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 499/986] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:12.968348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:12.968594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23f2e5-2d2a-41ff-8b7a-a342b06d8e68 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:13.010583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:13.011197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:13.011510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:13.011794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:13.012496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:13.013932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:13.020565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 488/987] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:13:13.034948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:13.087308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22621c66-2716-4819-b96e-bbb75e12e1ee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:13.126924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:13.127694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:13.127988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:13.128270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:13.129091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:13.129411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:13.130887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb0e3a-eee7-4b94-83b5-f860c7fe1f23 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:13.149219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 500/988] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:13.150809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:13.237989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:13.262957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd1a18-c0f9-4fcc-bc3a-701b8ead0566 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:13.297896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:13.298560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:13.298852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:13.299135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:13.300184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] RBAC: Adding query filter params (role.id=e093dc275c994361b21cb90aceb44ff2, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=cbf595eb1a0c44afb4d4f96413763631) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:13.300510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:13.302000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc75785-fb6f-436f-a49a-f40cb321362f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:13.308390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 489/989] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:13 2022] GET /identity/v3/role_assignments?user.id=cbf595eb1a0c44afb4d4f96413763631&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=e093dc275c994361b21cb90aceb44ff2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:14.885023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d83-16b1-46ae-8b83-5b73c9dcdf7f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:14.885572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d83-16b1-46ae-8b83-5b73c9dcdf7f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:14.885572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6d83-16b1-46ae-8b83-5b73c9dcdf7f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:14.886999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 501/990] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:14.892025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799a291-3acc-4c05-9739-0d8031d62419 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:14.892396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799a291-3acc-4c05-9739-0d8031d62419 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:14.892600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799a291-3acc-4c05-9739-0d8031d62419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:14.910154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b799a291-3acc-4c05-9739-0d8031d62419 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:14.911378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b799a291-3acc-4c05-9739-0d8031d62419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:14.980741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 490/991] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:15.002756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6d83-16b1-46ae-8b83-5b73c9dcdf7f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.053633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceeb84-22d1-42e2-b447-c95e00bd3176 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.053946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceeb84-22d1-42e2-b447-c95e00bd3176 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.054210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceeb84-22d1-42e2-b447-c95e00bd3176 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.055754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 491/992] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:15.060607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.060878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.061166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.078557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:15.079783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:15.110092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e6d83-16b1-46ae-8b83-5b73c9dcdf7f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.124775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.146152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.146331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 492/993] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:15.147398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.147872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.148330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.149169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.150733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:15.163387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:15.164386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:15.165243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.167429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 502/994] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:15.169703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.173220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.173716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.276198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a51dd-5399-401b-8870-1106ae12c5b9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.285271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec04-62e1-40ad-b048-8583ff0e4912 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.315094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.315976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.316259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.316586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.317250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.318618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.321158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.321829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.322118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.322400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.323118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:15.323472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.324827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf78544-143d-4349-99a8-78c10b792255 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.330246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 503/995] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:15.331197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:15.332205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 15:13:15.333075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:15.333931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114b7c7-5a25-444f-8e0c-251d5f465fb6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.333931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114b7c7-5a25-444f-8e0c-251d5f465fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.333931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114b7c7-5a25-444f-8e0c-251d5f465fb6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.334123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 493/996] 10.222.0.46 () {56 vars in 1126 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:15.335532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 504/997] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:15.335707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.338296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114b7c7-5a25-444f-8e0c-251d5f465fb6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.382053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.403147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.436938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16895c-6535-43ba-a46d-b7fa24dd1a9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.438849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114b7c7-5a25-444f-8e0c-251d5f465fb6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.470161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.470827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.471118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.471427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.471568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.472222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:15.472325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.472502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.472593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.472778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.473473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:15.473779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.473901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2e953-2645-47ed-b8cf-c3e7e242ad43 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.475114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.478902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 494/998] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:15.484523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 505/999] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:15.484658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.484658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.484658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.484909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.498488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:15.499702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:15.564700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 495/1000] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:15.565511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.588699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86044668-1f06-4f76-9c48-3aa2d18e75c4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.596244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.623726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.624329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.624446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.624662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.624945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.626084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] RBAC: Adding query filter params (interface=public, service_id=e694b081543442378a09a74544674a2b, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:15.626409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.627892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.633041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 506/1001] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/endpoints?service_id=e694b081543442378a09a74544674a2b&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:15.633739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.662781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce10b-5b92-4b62-9a58-4eb6793130f2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.696345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.697004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.697291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.697567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.698274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:15.698580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.699977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.705263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 496/1002] 10.222.0.46 () {56 vars in 1122 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:15.706013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.734265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8c1b-06e0-45cd-ab8a-993278c12193 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.767994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.768784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.769067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.769346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.769997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.771381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:15.783613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:15.784591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:15.790490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 507/1003] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:15.791345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.803667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696f7-4573-4afd-a5f0-5c2279ba064c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.824465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.830438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:15.839563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.840422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.840712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.840996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.841680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.843045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.849308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 497/1004] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:13:15.850077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:15.905106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290baa84-cb2d-4b9b-85ae-969c5b8817aa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:15.945801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:15.946500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:15.946785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:15.947177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:15.947970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:15.949370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:15.971411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 508/1005] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:15.971648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee282-f54d-4f87-920f-172d366a1b95 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:16.028918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:16.029645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:16.029941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:16.030220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:16.031098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:16.032596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:16.033679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd02359-7338-40fa-938e-280067beb1b7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:16.039373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 498/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:16.046219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:16.156794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:16.164672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45799d52-f48a-4806-9f8a-5c075a081961 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:16.200794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:16.201439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:16.201728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:16.202011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:16.202734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:16.203049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:16.204456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab66e2-1762-4f21-bae4-cb0942004c5f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:16.214958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 509/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:17.893060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667462e-b6da-43e8-83cf-896f1105b4bb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:17.893605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667462e-b6da-43e8-83cf-896f1105b4bb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:17.893605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667462e-b6da-43e8-83cf-896f1105b4bb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:17.895159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 499/1008] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:17.900104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:17.900511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:17.900676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:17.917720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:17.919161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:17.983343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf076-eb7b-4949-a279-743511b7f223 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:17.983659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf076-eb7b-4949-a279-743511b7f223 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:17.983918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf076-eb7b-4949-a279-743511b7f223 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:17.985283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 500/1009] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:17.989672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 510/1010] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:17.990474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:17.990763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:17.991044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.007505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:18.008681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:18.012085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.062648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.084343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 501/1011] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:18.104717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.107843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.125722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285f7d5-c58a-4fee-b0f9-a60c6b1dd8b4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.163121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.163873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.164517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.164517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.165084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.166452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:18.179056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:18.180213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:18.181080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 511/1012] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:18.187042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.217160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66901-7165-4e1a-b600-07f41b40a48a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.262060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.262860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.263150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.263498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.264687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.266132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4b9db-3452-4982-9517-2e0d6b8ad434 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.267792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 502/1013] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:18.268963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eae96-6184-424b-8df9-6b71963f4260 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.269423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eae96-6184-424b-8df9-6b71963f4260 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.269846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76eae96-6184-424b-8df9-6b71963f4260 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.275068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 503/1014] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:18.275832 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76eae96-6184-424b-8df9-6b71963f4260 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.290901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e7d1-0d26-490a-934c-48d7f17ef1cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.324969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.325668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.325987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.326279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.326478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.326989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:18.327330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.328698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad90ac2-0982-4d15-8b84-d238893eadd6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.333514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 512/1015] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:18.334873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.335174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.335485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.351621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:18.352814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:18.363853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.380484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76eae96-6184-424b-8df9-6b71963f4260 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.413605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.414242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.414495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.414780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.415514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.416957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.420356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 513/1016] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:18.421327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.445156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 504/1017] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:18 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:18.446365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.542377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120f1c-963d-4a59-a85b-4fefe38c507d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.562110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.565303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2943e6d-9cd2-49ce-b1df-f996e1040d91 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.584451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.585185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.585505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.585788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.586506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:18.586813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.588230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.599393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 514/1018] 10.222.0.46 () {56 vars in 1112 bytes} [Wed Jul 6 15:13:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:18.600707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:18.604848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.605523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.605808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.606091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.606881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.608878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeafc6-30dc-47b9-809f-caa739600672 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.616192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:18.627478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 505/1019] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:18 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:18.722793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83fb4d-78b5-48be-89b3-f677fc8a4e7e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:18.768152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:18.768836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:18.769127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:18.769410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:18.770166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:18.772338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78471946-64b5-4ceb-9a09-5f83d9a2dbc5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:18.793981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 515/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:20.535584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138809f-f7f6-4607-8f9e-3ac444ee7578 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.536090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138809f-f7f6-4607-8f9e-3ac444ee7578 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.536090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138809f-f7f6-4607-8f9e-3ac444ee7578 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.538065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 506/1021] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:20.543487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8ee9f-96e3-4ba8-9b30-a8d00ff926d3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.543934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8ee9f-96e3-4ba8-9b30-a8d00ff926d3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.544090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8ee9f-96e3-4ba8-9b30-a8d00ff926d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.560579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e1c8ee9f-96e3-4ba8-9b30-a8d00ff926d3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:20.561957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e1c8ee9f-96e3-4ba8-9b30-a8d00ff926d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:20.621230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:20.643266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 516/1022] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:20.665264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138809f-f7f6-4607-8f9e-3ac444ee7578 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:20.667815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb44ee-a4b6-40c8-8ae2-2ba522e35d4d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.668115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb44ee-a4b6-40c8-8ae2-2ba522e35d4d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.668655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb44ee-a4b6-40c8-8ae2-2ba522e35d4d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.670066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 517/1023] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:20.675071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.675398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.675679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.692270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:20.693446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:20.769278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 518/1024] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:20.777238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138809f-f7f6-4607-8f9e-3ac444ee7578 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:20.792174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:20.813328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:20.814126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.814408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.814683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.815874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:20.817305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:20.819049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 507/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:20.824524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:20.852706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:20.908302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d76512-573f-4e5c-9f22-16f115592c94 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:20.925698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:20.927673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987790a-8fa0-433f-96b7-77e5c1348e71 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:20.945789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:20.946580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.946861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.947134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.947816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:20.949173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3a69aab7-80ef-449f-92c1-af66498678fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:20.961212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:20.962349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:20.963269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 519/1026] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 6 15:13:20 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:20.964110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056321-7ce5-42c8-91f1-61164e435f8d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.964801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056321-7ce5-42c8-91f1-61164e435f8d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.964801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056321-7ce5-42c8-91f1-61164e435f8d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.965817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 520/1027] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:20.968280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056321-7ce5-42c8-91f1-61164e435f8d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:20.968408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:20.969081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.969375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.969656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:20.970383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:20.971874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520aec9b-0962-4c72-9135-88f70418aae2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:20.993465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 508/1028] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:13:20 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:20.994716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:20.995016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:20.995327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.026920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:21.028454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:21.089250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056321-7ce5-42c8-91f1-61164e435f8d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.108251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 509/1029] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:21.129608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.130556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.131031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.132132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.134427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.135270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:21.135601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.137148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.143735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 521/1030] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:21.148928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.149686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.206702 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.250976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee23662-3b2e-4da8-b010-9c5c4994c62f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.267907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8e0ae-4d3c-4749-a83a-6116865b3e59 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.286924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.287981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.289673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.289673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.301173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:21.308898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:21.309504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:21.311299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:21.312169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:21.312834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.312834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.312834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.313019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:21.313447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.314662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.316207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 510/1031] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:21.317252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.325559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 522/1032] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:21.325815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.400251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.427978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003902c-24e5-4574-a5e1-d3cd91bc5c78 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.430835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef773da3-9286-4a16-86b0-86eaebfaa3f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.442708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.464565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.465358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.465708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.465993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.467304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:21.467626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.469005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.471566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.472403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.472838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.473267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.474229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] RBAC: Adding query filter params (interface=public, service_id=11a32a33d0df4ee2acf4d5686533c86d, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:21.474552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.475927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.482562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 511/1033] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:21.483654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.486435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 523/1034] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/endpoints?service_id=11a32a33d0df4ee2acf4d5686533c86d&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:21.491505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.591837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14606ec-59de-4edc-ab01-beed0f26694f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.597372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ffd91-89fa-490b-8c80-255c878859fb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.626695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.627622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.628028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.628308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.631359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.632996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.633742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.634032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.634352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.635045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.636507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.640484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.641907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.651568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 524/1035] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:21.670631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:21.671659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:21.672629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:21.672629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:21.672629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:21.672629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:21.675596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 512/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:21.677760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.723297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:21.788896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9171a1-2a74-44b9-b593-c28ff012848c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.824033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.824711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.825001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.825281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.826544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:21.826858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.828310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:21.840264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 525/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:21.846504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:21.947979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750212a6-0dc1-48ae-abd2-08dd42a7b358 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:21.981556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:21.982381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:21.982669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:21.982950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:21.990448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:21.991921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:22.006079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:22.007030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:22.013010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 513/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:13:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:22.015498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:22.114790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:22.119157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01008d-7d8e-4e12-a9b4-6fc72d8b7069 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:22.160671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:22.161661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:22.162089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:22.162602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:22.163650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:22.164138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:22.165711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.184127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 526/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:13:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:22.189988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:22.244838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:22.301006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240bc35-8e82-40ae-ae58-aeac7d96e7c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:22.335885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:22.336544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:22.336832 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:22.337112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:22.403707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:22.405187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.452320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:22.504346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 514/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:22.509729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:22.609330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c16bc5-11aa-4089-bba3-151326fa3cb6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:22.643271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:22.644077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:22.644366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:22.644648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:22.654685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:22.656163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:22.674260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:22.675159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:22.676388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:22.676388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:22.676388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:22.676388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:22.679532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 527/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:22.681052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:22.782267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad62f9-a422-46e4-ae0d-680ad769c303 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:22.816883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:22.817543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:22.817826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:22.818106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:22.818822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:22.819130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:22.820510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.836296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 515/1042] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:22.837642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:22.849007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:22.937587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c73275-2de5-4db8-af7c-db4c81a496a2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:22.970894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:22.971719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:22.971998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:22.972276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:22.978786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:22.980257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b09a5a6a-4aa4-4391-9360-8903eb54bffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:22.994147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:22.995104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:22.996062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 528/1043] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:23.002556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.014946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.078679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c8e-459d-476b-baa3-358fc37bf83f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.078993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c8e-459d-476b-baa3-358fc37bf83f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.079173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff158c8e-459d-476b-baa3-358fc37bf83f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.080706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 529/1044] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:13:23.085463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.085756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.086036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.103042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:23.104257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:23.109140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee21e9-e7d3-44c9-91c4-7bacf4ff09da None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.142865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.143560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.143849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.144130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.144990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:23.145308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.147258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.161179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 516/1045] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:23.167046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.167765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.174614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 530/1046] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:23.196916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.236562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.280266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623fbf3-3f49-48e4-a6ca-386c3d769346 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.309836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e042405-8e92-44d6-ad0a-829f5de1a63a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.314284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.314946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.315243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.315538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.316372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] RBAC: Adding query filter params (role.id=e093dc275c994361b21cb90aceb44ff2, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:23.316689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.318119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.331357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 517/1047] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=e093dc275c994361b21cb90aceb44ff2 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:23.347778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.348599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.348890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.349166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.361478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:23.369410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:23.370578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:23.371763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:23.375407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 531/1048] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:23.376931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.484179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9a42-745b-414c-8117-be9f0074a91a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.513949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.519689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.520752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.521431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.521715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.522469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:23.522779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.524172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc393fc1-a3b3-4189-9e55-1691f646960d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.538305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 518/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:23.539912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.599982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.643826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c8dc-86ed-4c29-8cda-fcd450d57979 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.677309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.678179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.678458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.678729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] PATH_INFO: `/v3/users/e2347c63da4a470da64a5a21af52b557` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.690827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc464af-3b8b-462e-8165-33c268ed4160 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.691334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc464af-3b8b-462e-8165-33c268ed4160 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.691448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc464af-3b8b-462e-8165-33c268ed4160 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.692948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 519/1050] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:23.693071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e2347c63da4a470da64a5a21af52b557)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.694506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.700553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 532/1051] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/users/e2347c63da4a470da64a5a21af52b557 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:23.701338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.701338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.701338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.702760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.715589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:23.716807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:23.785756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 520/1052] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:23.807614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:23.814111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e5b46-badd-44f6-ac5f-6737d9f6a7c4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.836042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.859422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.860454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.860746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.861029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.911863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ccb4-9732-4b89-bbb4-1d3b92af608b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:23.932053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.933609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.937775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:23.949908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:23.951298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:23.951592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:23.951873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] PATH_INFO: `/v3/services/image` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:23.952660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:23.954164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:23.967083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:23.968153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:23.969005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 521/1053] 10.222.0.46 () {56 vars in 1105 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:23.974367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:24.038264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 533/1054] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 15:13:24.043968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:24.065325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:24.089088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0622fda-25d4-4f01-a2c4-6fc8da260e04 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:24.124142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:24.124752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:24.125129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:24.125417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:24.126134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] RBAC: Adding query filter params (name=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:24.126501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:24.127862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:24.137171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 522/1055] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:24.138251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:24.155978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca1a5-02ec-424a-ba3e-c961994bf228 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:24.193659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:24.196358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:24.197251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:24.197560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:24.197839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:24.198992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:24.200450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:24.202143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 534/1056] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:24 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:24.208380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:24.250774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4485e-bf12-49ef-b36d-6cee15bb4e9f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:24.287159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:24.288153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:24.288654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:24.289081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:24.289940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] RBAC: Adding query filter params (type=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:24.290465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:24.292140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:24.297991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:24.299464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 523/1057] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:24.304728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:24.319367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b82c21-885a-457d-b8c3-b886f6210093 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:24.352829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:24.353524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:24.353809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:24.354086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:24.354933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6, user.id=e2347c63da4a470da64a5a21af52b557) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:24.355283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:24.356740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95b646-f1ad-4021-be54-89fafbbb0e80 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:24.362927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 535/1058] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:24 2022] GET /identity/v3/role_assignments?user.id=e2347c63da4a470da64a5a21af52b557&scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:24.428672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ececb-c210-4048-b91e-40090ad19c00 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:24.446895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:24.467651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:24.468334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:24.468620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:24.468897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:24.469655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:24.471043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f4e9f-8444-4657-a8c9-42076d6f1392 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:24.499274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 524/1059] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:13:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:25.332974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f626-ae29-4921-8f7f-07abbec7912e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:25.333506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f626-ae29-4921-8f7f-07abbec7912e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:25.333599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051f626-ae29-4921-8f7f-07abbec7912e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:25.334947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 536/1060] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:25.340061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35cbd-1989-411b-a667-3920462e319d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:25.340454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35cbd-1989-411b-a667-3920462e319d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:25.340619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35cbd-1989-411b-a667-3920462e319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:25.358015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-33c35cbd-1989-411b-a667-3920462e319d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:25.359186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33c35cbd-1989-411b-a667-3920462e319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:25.429652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 525/1061] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:25.452108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051f626-ae29-4921-8f7f-07abbec7912e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:25.554762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051f626-ae29-4921-8f7f-07abbec7912e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:25.592778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:25.593850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:25.594284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:25.594712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:25.605209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:25.613845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:25.614954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:25.615848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:25.616492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 537/1062] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:25.621883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35cbd-1989-411b-a667-3920462e319d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:25.717972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:25.722262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35cbd-1989-411b-a667-3920462e319d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:25.757948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:25.758645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:25.758934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:25.759228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:25.759995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:25.760307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:25.761684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:25.774569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 526/1063] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:25.776031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:25.787987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:25.879086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8f0be-3f92-4267-af17-7ca12137a7c9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:25.912575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:25.913370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:25.913657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:25.913937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:25.924718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:25.926158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:25.944895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:25.945817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:25.946733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:25.946733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:25.946733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:25.946733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:25.946733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 538/1064] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:25.952611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.053457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344d77d9-8012-43f1-ab63-7e08ec7d7b1f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.087726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.088400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.088690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.088971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.089698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:26.090008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.091435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.104134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 527/1065] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:26.109494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.169743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.212140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5f0c-ebb2-4237-9284-458df7fbc9f7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.256239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.257042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.257327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.257599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.265468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.266914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-bd73e6b5-c11a-498f-be35-959406a7f80f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:26.281040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:26.282002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:26.282966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 539/1066] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:26.289367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.313270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.379604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13504-48ae-41a9-87b0-24a1efe44fa8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.380157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13504-48ae-41a9-87b0-24a1efe44fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.380157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13504-48ae-41a9-87b0-24a1efe44fa8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.381514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 540/1067] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:13:26.387573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.387872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.388196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.393654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b88-d236-4669-a31c-eef5959a9e4f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.405063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:26.406261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:26.429605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.430517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.430984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.431294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.432140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:26.432449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.433866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.456536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 528/1068] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:26.457550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.487536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 541/1069] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:26.505386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.539969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.557725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.565244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b328-a46e-45fd-bf6a-ddd2a5b070ae None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.600865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.601663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.601956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.602239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.608089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232712c6-a570-4456-b4a4-ee2302b1caaf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.612884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.614339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5df25466-0232-45a3-abfb-8f2f6df70697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:26.631427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:26.632352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:26.633273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:26.633273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:26.633273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:26.633273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:26.633273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 529/1070] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:26.633801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462267d7-7803-4786-9a82-053a56cab068 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.634085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462267d7-7803-4786-9a82-053a56cab068 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.634361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462267d7-7803-4786-9a82-053a56cab068 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.635630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 530/1071] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:26.639446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462267d7-7803-4786-9a82-053a56cab068 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.646107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.646913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.647201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.647518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.658443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ee95466a-ef11-437a-bf02-4fce22e0e581 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:26.666065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:26.667125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:26.668496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 542/1072] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:26.669871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.669871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.669871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.687257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:26.688400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:26.740661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462267d7-7803-4786-9a82-053a56cab068 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.753336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 543/1073] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:26.754375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.770601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.777667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.778338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.779113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.779427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.780155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:26.780465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.781836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.793552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 531/1074] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:26.795004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.796914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.853713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497914-9986-4b57-b7fb-266856f38979 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.887066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.887947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.888234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.888511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.889256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:26.889566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.890938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.893429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb9e1-ef03-4eb9-b869-bddcb49ecea6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:26.898904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 544/1075] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:26.899705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:26.927737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:26.928383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:26.928691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:26.928965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:26.984921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:26.995879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:26.997339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:26.998881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcda9e-f0c2-41e3-be24-29ca4f44a6d8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.000420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:27.033523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.034310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.034594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.034867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.041377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.042800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:27.056848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:27.057743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 545/1076] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:27.058937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.097301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 532/1077] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:27.098126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.160711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32671994-5b5b-4087-b3a2-034955ca2460 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.184367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:27.193406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b6502-613b-4e04-8b00-5e9d381418e0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.195548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.196487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.196776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.197051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] PATH_INFO: `/v3/users/e2347c63da4a470da64a5a21af52b557` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.198247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] RBAC: Authorizing `identity:get_user(user_id=e2347c63da4a470da64a5a21af52b557)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.199717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.201896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 546/1078] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:27 2022] GET /identity/v3/users/e2347c63da4a470da64a5a21af52b557 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:27.202628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.231409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.232068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.232348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:27.232464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.232621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.233414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:27.233783 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.235515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.247648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 533/1079] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:27.248490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.305057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3e314-3cb0-445c-a617-954f0f8881a7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.338212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.339005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.339317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.339598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.347530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe170ca-2248-4262-b31f-0d9f5ee3ec27 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.350118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.351595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:27.368912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:27.369818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:27.370714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:27.370714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:27.370714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:27.370714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:27.370714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 547/1080] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:27.371005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.382565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.383468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.383755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.384033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.385214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.386659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.388989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 534/1081] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:27 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:27.389732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.412521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:27.453260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:27.471769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c0dd8-f8ab-4d7b-b4cc-345739e9fdd4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.491531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59727-bb1a-476f-a59c-cb5c4702d62d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.505084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.505976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.506259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.506535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/e093dc275c994361b21cb90aceb44ff2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.508761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=aff09cb4b1ff42c0a011f8e09a3c6c38, role_id=e093dc275c994361b21cb90aceb44ff2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.510186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.523709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 548/1082] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:13:27 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/aff09cb4b1ff42c0a011f8e09a3c6c38/roles/e093dc275c994361b21cb90aceb44ff2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:13:27.524447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:27.526201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.527258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.527798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.528134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.528889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:27.529200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.530699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82992b-dd0b-4060-a9b2-c2076161b531 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.551776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 535/1083] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:27.647035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2df6d-77e3-4b07-9f0c-f46594410510 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:27.688543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:27.688543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:27.688543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:27.688881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6/users/e2347c63da4a470da64a5a21af52b557/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:27.690946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] RBAC: Authorizing `identity:create_grant(project_id=374799e8d2e8435fb42854ae7bd6c6f6, user_id=e2347c63da4a470da64a5a21af52b557, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:27.692450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b84c89-4e69-4b56-941e-258b6698592b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:27.708570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 549/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:13:27 2022] PUT /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6/users/e2347c63da4a470da64a5a21af52b557/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:13:29.549177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc84f6-a8c0-4054-b894-7663bb068525 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.549670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc84f6-a8c0-4054-b894-7663bb068525 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.549775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc84f6-a8c0-4054-b894-7663bb068525 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.551097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 536/1085] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:29.556258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe6df0-aa8e-49e0-be44-4847068c1bd2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.556654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe6df0-aa8e-49e0-be44-4847068c1bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.556815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe6df0-aa8e-49e0-be44-4847068c1bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.562418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:29.576015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e6fe6df0-aa8e-49e0-be44-4847068c1bd2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:29.577208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e6fe6df0-aa8e-49e0-be44-4847068c1bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:29.642425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 550/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:29.666176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc84f6-a8c0-4054-b894-7663bb068525 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:29.692046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece0f3-e0b2-4c5d-97d3-0ffc7964f48f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.692334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece0f3-e0b2-4c5d-97d3-0ffc7964f48f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.692621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eece0f3-e0b2-4c5d-97d3-0ffc7964f48f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.693978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 551/1087] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:13:29.698806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.699103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.699414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.715368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:29.717025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:29.746032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:29.768268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc84f6-a8c0-4054-b894-7663bb068525 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:29.783188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 552/1088] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:29.802117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:29.802867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.803149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.803450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.807152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:29.813230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:29.821355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:29.822289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:29.823144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 537/1089] 10.222.0.46 () {56 vars in 1120 bytes} [Wed Jul 6 15:13:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:29.828261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:29.870046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:29.908399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924586b-2a7e-4f99-8155-b6c002d72271 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:29.927554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73676c70-2169-4b70-bc81-32d2ac1d486d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:29.943650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:29.944442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.944723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.945000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.955737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:29.961443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:29.962105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:29.962388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:29.962662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:29.963332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:29.964390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:29.965368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:29.966558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:29.966558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:29.966695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 553/1090] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:29.966763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:29.970995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:29.971294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:29.975083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 538/1091] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:29.980145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.069796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11edd462-0292-468a-b0ac-7fc7f48de3c1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.076252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437fb09-fa23-4ade-8617-57301624b342 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.093181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.104857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.105518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.105802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.106081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.106853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:30.107160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.108557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.108978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.109765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.110050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.110328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.116622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 554/1092] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:30.120589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.122022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.122191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:30.139236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:30.140250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:30.141153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:30.141153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:30.141153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:30.141153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:30.141153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 539/1093] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:30.146988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.198499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.225960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056dfd-dfa5-49e2-9443-6435a0bcfa61 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.246114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1aad9-9a63-430e-8ffb-f707c58d7136 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.264151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.265032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.265320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.265597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] PATH_INFO: `/v3/users/e2347c63da4a470da64a5a21af52b557` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.266810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] RBAC: Authorizing `identity:get_user(user_id=e2347c63da4a470da64a5a21af52b557)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.268280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.269925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 555/1094] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/users/e2347c63da4a470da64a5a21af52b557 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:30.275727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.280554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.281220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.281509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.281788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.282803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:30.283121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.284520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.296808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 540/1095] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:30.302566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.328278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.375565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240e6ef-65a9-4b8d-949d-5843c9681c1b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.400322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8d8e2-49df-4ccf-a43b-3f46827d1e51 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.408405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.409058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.409349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.409630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.413243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.434600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.435965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.436259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.436538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.443547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.444987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:13:30.459772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:30.460945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:30.461932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 541/1096] 10.222.0.46 () {56 vars in 1093 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:30.466754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.478944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.480436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.530569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.564319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d7239-942e-4641-8447-e51d6296ad90 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.580406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 556/1097] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 15:13:30.586045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.598579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.599284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.599570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.599847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.600690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:30.601001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.602431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.619851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 542/1098] 10.222.0.46 () {56 vars in 1133 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:30.626004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.654594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.683828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83155e-c316-4ecc-ba3b-5b450c099d94 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.717638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.718510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.718834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.719116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.720389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.721806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.721969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d830d0b-fc83-4fe0-b465-0321283e88a4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.723529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 557/1099] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:30.729290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:30.750767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.756085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.756738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.757022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.757307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.824673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.826134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.828568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce61-8313-463b-bbac-5b4a7aa16e72 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:30.854439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:30.861586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:30.862230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:30.862515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:30.862789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:30.863650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6, user.id=e2347c63da4a470da64a5a21af52b557) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:30.863965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:30.865371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:30.871188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 558/1100] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/role_assignments?user.id=e2347c63da4a470da64a5a21af52b557&scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:13:30.928622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 543/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 15:13:30.935408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:31.021662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:31.037840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f469667-f48f-4210-a625-9bf4bab24a41 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:31.072691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:31.073496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:31.073782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:31.074083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:31.084735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:31.086174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:31.103764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:31.104707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:31.105661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:13:31.105661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:31.105661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:13:31.105661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:31.105661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 559/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:13:31.111130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:31.206259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e600a-b942-41ac-a415-ffe15a8a065e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:31.234411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:31.240417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:31.241095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:31.241383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:31.241661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:31.242393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:31.242704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:31.244319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:31.256308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 544/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:31.261810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:31.360059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69aeca-9512-4d26-b195-75caa6f69b33 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:31.391761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:31.392549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:31.392833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:31.393109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:31.395743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:31.400194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:31.401787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:13:31.416124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:31.417193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:31.418372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 560/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:31.423804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:31.520285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744845f-700f-4521-826d-343e95add6f3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:31.553676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:31.554340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:31.554625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:31.554904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:31.555728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:31.556042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:31.557478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b3d74-e380-45a0-8f29-5c475b8e1acd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:31.573927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 545/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:31.576264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:31.676722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65ab4b-af65-433f-88a5-756c67a4fdee None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:31.709616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:31.710262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:31.710548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:31.710826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:31.711694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] RBAC: Adding query filter params (role.id=e093dc275c994361b21cb90aceb44ff2, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=aff09cb4b1ff42c0a011f8e09a3c6c38) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:31.712013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:31.713431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ea56-2f42-4867-aa00-6c36b7d098cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:31.719951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 561/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:31 2022] GET /identity/v3/role_assignments?user.id=aff09cb4b1ff42c0a011f8e09a3c6c38&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=e093dc275c994361b21cb90aceb44ff2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:32.905775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef4a9-ca46-428d-8903-8ac6e5a2ec6a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:32.906317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef4a9-ca46-428d-8903-8ac6e5a2ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:32.906317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ef4a9-ca46-428d-8903-8ac6e5a2ec6a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:32.907747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 546/1107] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:32.912877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:32.913281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:32.913436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:32.930458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:32.931668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:32.984827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:32.996772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 562/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:13:33.021265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ef4a9-ca46-428d-8903-8ac6e5a2ec6a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:33.071046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:33.119413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ef4a9-ca46-428d-8903-8ac6e5a2ec6a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:33.152043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:33.152817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:33.153099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:33.153959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:33.155115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:33.156605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd5783-aa12-4543-b2b7-95d8497fa84a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:33.158121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 547/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:33.163791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:33.258816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78e166-3b5c-4374-a2fb-6736a28e2a0b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:33.292286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:33.293216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:33.293508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:33.293785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:33.294488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:33.295946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2f6c-5780-4ab8-956c-ac3d54766f2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:33.316213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 563/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:13:33 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:35.344708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7b13-9e8d-40e0-8463-108b80b38813 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:35.345199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7b13-9e8d-40e0-8463-108b80b38813 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:35.345291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7b13-9e8d-40e0-8463-108b80b38813 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:35.346601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 548/1111] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:35.351677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:35.352042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:35.352247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:35.369109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:35.370302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:35.410865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:35.436621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 564/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:35.459493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a7b13-9e8d-40e0-8463-108b80b38813 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:35.556551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a7b13-9e8d-40e0-8463-108b80b38813 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:35.573246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:35.589155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:35.589999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:35.590282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:35.590556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] PATH_INFO: `/v3/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:35.592544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] RBAC: Authorizing `identity:get_role(role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:35.593905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:35.595239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 549/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:35 2022] GET /identity/v3/roles/7a0d559703ba47979a34ed053388473e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:35.600726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:35.696417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3332cab-b6e2-4a8c-9177-d89d3323a47d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:35.729095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:35.729960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:35.730249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:35.730533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] PATH_INFO: `/v3/users/40de7a54eabb482e980b734368c6ed36` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:35.743766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] RBAC: Authorizing `identity:get_user(user_id=40de7a54eabb482e980b734368c6ed36)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:35.745197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:35.746818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 565/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:35 2022] GET /identity/v3/users/40de7a54eabb482e980b734368c6ed36 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:35.752734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:35.848044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376873-323e-4574-b12c-cdaa20f9fb41 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:35.879880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:35.880507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:35.880791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:35.881071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:35.933495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:35.947417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:35.948855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:36.041977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 550/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:36.048507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:36.081607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:36.145692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e552a0-1969-4e42-b4e4-0051a5fafeff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:36.178793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:36.179669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:36.179958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:36.180237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:36.181411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:36.182823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4888b-0387-4c65-8b71-f3a7156bdd58 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:36.185068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 566/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:36 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:36.190929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:36.274266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:36.288893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b157f-1a23-49c1-9783-b918fbf88b50 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:36.328153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:36.328810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:36.329094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:36.329366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:36.330200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6, user.id=40de7a54eabb482e980b734368c6ed36) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:36.330510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:36.331964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a541f4-109c-4b33-bbe2-f741ae9a36fc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:36.337506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 551/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:36 2022] GET /identity/v3/role_assignments?user.id=40de7a54eabb482e980b734368c6ed36&scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6&role.id=7a0d559703ba47979a34ed053388473e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:38.288532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6e828-5cac-483c-b1a6-de6ca85cb151 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.289050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6e828-5cac-483c-b1a6-de6ca85cb151 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.289147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a6e828-5cac-483c-b1a6-de6ca85cb151 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.290464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 567/1118] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:38.295937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.296333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.296509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.312782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:38.313955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:38.378264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 552/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:38.400648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6e828-5cac-483c-b1a6-de6ca85cb151 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:38.497698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a6e828-5cac-483c-b1a6-de6ca85cb151 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:38.499173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:38.531765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:38.532547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.532831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.533104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] PATH_INFO: `/v3/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.534546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] RBAC: Authorizing `identity:get_role(role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:38.535919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:38.537253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 568/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:38 2022] GET /identity/v3/roles/7a0d559703ba47979a34ed053388473e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:38.542781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:38.619777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:38.639444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d125260-f114-40ca-b58a-7b726a030313 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:38.671209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:38.672112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.672396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.672674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] PATH_INFO: `/v3/users/40de7a54eabb482e980b734368c6ed36` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.673843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] RBAC: Authorizing `identity:get_user(user_id=40de7a54eabb482e980b734368c6ed36)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:38.675290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:38.676882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 553/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:38 2022] GET /identity/v3/users/40de7a54eabb482e980b734368c6ed36 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:38.682484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:38.777093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b256d-e393-4d4d-a0db-b20226d83441 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:38.810132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:38.810914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.811192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.811493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.812647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:38.814061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9cb8e-1c07-45e8-b578-08beb374f270 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:38.815743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 569/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:38 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:38.821349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:38.917369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef6031-e85f-4e03-9295-9b7e107ba87c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:38.950861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:38.951763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:38.952048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:38.952324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6/users/40de7a54eabb482e980b734368c6ed36/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:38.954507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] RBAC: Authorizing `identity:create_grant(project_id=374799e8d2e8435fb42854ae7bd6c6f6, user_id=40de7a54eabb482e980b734368c6ed36, role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:38.955993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1cecb-a5aa-41ce-b950-d7e4d2752002 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:38.968343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 554/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:13:38 2022] PUT /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6/users/40de7a54eabb482e980b734368c6ed36/roles/7a0d559703ba47979a34ed053388473e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:13:40.946531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d54f8-5570-43b0-be49-06b8dc97f23b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:40.947105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d54f8-5570-43b0-be49-06b8dc97f23b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:40.947105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d54f8-5570-43b0-be49-06b8dc97f23b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:40.948479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 570/1124] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:40.953515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:40.953945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:40.954082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:40.970939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:40.972162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:40.974092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:41.038609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 555/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:41.062865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d54f8-5570-43b0-be49-06b8dc97f23b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:41.101768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:41.159884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d54f8-5570-43b0-be49-06b8dc97f23b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:41.193031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:41.193873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:41.194156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:41.194431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] PATH_INFO: `/v3/roles/7a0d559703ba47979a34ed053388473e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:41.195908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] RBAC: Authorizing `identity:get_role(role_id=7a0d559703ba47979a34ed053388473e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:41.197304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:41.198588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 571/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:41 2022] GET /identity/v3/roles/7a0d559703ba47979a34ed053388473e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:41.204190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:41.300477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb1160-0f26-46b5-a101-22adf7925994 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:41.332542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:41.333412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:41.333696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:41.333972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] PATH_INFO: `/v3/users/40de7a54eabb482e980b734368c6ed36` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:41.335151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] RBAC: Authorizing `identity:get_user(user_id=40de7a54eabb482e980b734368c6ed36)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:41.336612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:41.338210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 556/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:41 2022] GET /identity/v3/users/40de7a54eabb482e980b734368c6ed36 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:41.344390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:41.439042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e00a53-948c-4c2c-ad07-c4db726b9d4f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:41.446073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:41.472474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:41.473117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:41.473402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:41.473680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:41.537241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:41.538683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:41.632072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 572/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:41.638514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:41.647071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:41.734563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc883bd-a1e9-4997-b57e-a6ed32b86ad5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:41.766276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:41.767104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:41.767553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:41.767834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] PATH_INFO: `/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:41.768992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] RBAC: Authorizing `identity:get_project(project_id=374799e8d2e8435fb42854ae7bd6c6f6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:41.770410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b24cb-0771-491f-be63-e52e725b0151 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:41.772082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 557/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:41 2022] GET /identity/v3/projects/374799e8d2e8435fb42854ae7bd6c6f6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:41.777877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:41.786420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:41.873739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff0ceb0-9a48-48d8-87ee-66dd037ccf6b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:41.906384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:41.907026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:41.907337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:41.907618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:41.908453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] RBAC: Adding query filter params (role.id=7a0d559703ba47979a34ed053388473e, scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6, user.id=40de7a54eabb482e980b734368c6ed36) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:41.908770 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:41.910184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9c89b-20bc-4729-a67f-3bce68e40cff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:41.915868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 573/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:41 2022] GET /identity/v3/role_assignments?user.id=40de7a54eabb482e980b734368c6ed36&scope.project.id=374799e8d2e8435fb42854ae7bd6c6f6&role.id=7a0d559703ba47979a34ed053388473e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:43.924040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297db-65d7-49ec-99f8-398ecdefa536 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:43.924566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297db-65d7-49ec-99f8-398ecdefa536 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:43.924653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297db-65d7-49ec-99f8-398ecdefa536 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:43.925965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 558/1131] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:43.931193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:43.931627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:43.931925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:43.949757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:43.950953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:44.009964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:44.015633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 574/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:44.038387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58297db-65d7-49ec-99f8-398ecdefa536 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:44.112245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:44.137219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58297db-65d7-49ec-99f8-398ecdefa536 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:44.170521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:44.171319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:44.171604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:44.171879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:44.173032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:44.174499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8524d8-c136-4566-8e64-ec57a564df5c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:44.176042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 559/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:44.181582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:44.281032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849a494c-f963-4f29-b64c-6215d09ae9ba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:44.315690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:44.316636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:44.317146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:44.317431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:44.318161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:44.319651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefdd69-f53b-4dbb-8fde-2b2b88560cc3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:44.345753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 575/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:13:44 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:13:46.353705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa84-bf28-42c1-94dc-debcde870c13 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:46.354238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa84-bf28-42c1-94dc-debcde870c13 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:46.354340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fa84-bf28-42c1-94dc-debcde870c13 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:46.355639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 560/1135] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:46.360919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:46.361330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:46.361482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:46.377973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:46.379171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:46.425189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:46.445254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 576/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:46.470489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fa84-bf28-42c1-94dc-debcde870c13 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:46.567161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fa84-bf28-42c1-94dc-debcde870c13 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:46.599373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:46.600152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:46.600439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:46.600719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:46.601877 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:46.603325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15504c7b-a475-42ff-8165-2febc2afcf4d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:46.604892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 561/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:46.610386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:46.705493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f612418-0e3d-4b94-9e9b-32a6eccf332e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:46.737093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:46.737747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:46.738032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:46.738308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:46.739002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:46.740451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64740610-ac83-4d8b-95f3-1b79aac7f4d6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:46.759904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 577/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:13:46 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:13:48.785276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c36142-f5f5-4ce2-894f-200844c6d22e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:48.785820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c36142-f5f5-4ce2-894f-200844c6d22e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:48.785820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c36142-f5f5-4ce2-894f-200844c6d22e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:48.787176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 562/1139] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:48.792221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:48.792901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:48.793056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:48.809970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:48.811153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:48.839467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:48.876855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 578/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:48.900372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c36142-f5f5-4ce2-894f-200844c6d22e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:48.912859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:48.998982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c36142-f5f5-4ce2-894f-200844c6d22e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.030971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.031775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.032059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.032334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.042034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:49.050352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:49.051572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:49.052848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 563/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:49.057379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:49.151196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd308ccd-4edf-4b47-a27f-172611c6ab5e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.184062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.184721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.185004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.185280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.186020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:49.186327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:49.187717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:49.195382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 579/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:49.200710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:49.266622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:49.298057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bc929-702d-4fa8-84b4-21cc508a168c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.329320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.330173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.330453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.330723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] PATH_INFO: `/v3/users/2c1f8f887aa640e79aed3e95d0127de6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.343685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1f8f887aa640e79aed3e95d0127de6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:49.345115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:49.346717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 564/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:49 2022] GET /identity/v3/users/2c1f8f887aa640e79aed3e95d0127de6 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:49.352382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:49.382645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:49.450488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab20c1a-e940-473a-97a5-74ce1bdd864d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.483922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.484565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.484855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.485137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.550935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:49.552350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:49.580594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:49.647175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 580/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:49.653595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:49.747876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72927aba-8646-415e-a79f-90aa246e0e19 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.773055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:49.781181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.782035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.782318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.782591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] PATH_INFO: `/v3/projects/3871816c83864bd4ab85314ff1db04a1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.783768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] RBAC: Authorizing `identity:get_project(project_id=3871816c83864bd4ab85314ff1db04a1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:49.785188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073df228-6ab0-4532-ba19-b4bfe8ac405b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:49.786820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 565/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:49 2022] GET /identity/v3/projects/3871816c83864bd4ab85314ff1db04a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:49.793055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:49.887904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bc81d-6e42-45e2-849d-f5990a93bf92 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:49.914978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:49.921437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:49.922080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:49.922362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:49.922637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:49.923492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=3871816c83864bd4ab85314ff1db04a1, user.id=2c1f8f887aa640e79aed3e95d0127de6) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:49.923810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:49.925214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85665e-7879-49b9-bfe9-a63c43f109ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:49.930502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 581/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:49 2022] GET /identity/v3/role_assignments?user.id=2c1f8f887aa640e79aed3e95d0127de6&scope.project.id=3871816c83864bd4ab85314ff1db04a1&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:51.931962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48548-c2f9-4465-9651-c30a67b614b1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:51.932484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48548-c2f9-4465-9651-c30a67b614b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:51.932596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e48548-c2f9-4465-9651-c30a67b614b1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:51.933923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 566/1147] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:51.939184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:51.939662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:51.939774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:51.956558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:51.957731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:52.021348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 582/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:52.043875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e48548-c2f9-4465-9651-c30a67b614b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:52.142638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e48548-c2f9-4465-9651-c30a67b614b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:52.175234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:52.176032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:52.176322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:52.176607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:52.187040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:52.194617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:52.195735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:52.196782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 567/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:52.202298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:52.298110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a59fae-5c0d-4410-a4cd-46acd8b8797f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:52.322715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:52.334430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:52.335097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:52.335417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:52.335697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:52.336447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:52.336757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:52.338110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:52.345983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 583/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:52.351714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:52.406584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:52.450440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a669cf9-21b6-4b24-9ed4-c1ab826e6093 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:52.482585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:52.483472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:52.483764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:52.484047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] PATH_INFO: `/v3/users/2c1f8f887aa640e79aed3e95d0127de6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:52.485228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] RBAC: Authorizing `identity:get_user(user_id=2c1f8f887aa640e79aed3e95d0127de6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:52.486657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:52.488283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 568/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:52 2022] GET /identity/v3/users/2c1f8f887aa640e79aed3e95d0127de6 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:52.493994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:52.589924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0d65-3ac2-4a1e-851f-e30dbb3610ca None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:52.622446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:52.623303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:52.623585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:52.623854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] PATH_INFO: `/v3/projects/3871816c83864bd4ab85314ff1db04a1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:52.625010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] RBAC: Authorizing `identity:get_project(project_id=3871816c83864bd4ab85314ff1db04a1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:52.626439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284f17c-a4d6-4a71-900a-dae9ab7c672e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:52.628113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 584/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:52 2022] GET /identity/v3/projects/3871816c83864bd4ab85314ff1db04a1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:52.633800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:52.729586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bebc3-25c6-4ced-838e-865b58a64b6d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:52.752258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:52.762872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:52.763781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:52.764063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:52.764336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] PATH_INFO: `/v3/projects/3871816c83864bd4ab85314ff1db04a1/users/2c1f8f887aa640e79aed3e95d0127de6/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:52.766251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] RBAC: Authorizing `identity:create_grant(project_id=3871816c83864bd4ab85314ff1db04a1, user_id=2c1f8f887aa640e79aed3e95d0127de6, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:52.767696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28bf5-b467-43c7-811e-5444174ede39 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:52.780573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 569/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:13:52 2022] PUT /identity/v3/projects/3871816c83864bd4ab85314ff1db04a1/users/2c1f8f887aa640e79aed3e95d0127de6/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:13:54.753618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ec93d-9251-4ee0-b133-c5856c4f6161 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:54.754149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ec93d-9251-4ee0-b133-c5856c4f6161 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:54.754250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ec93d-9251-4ee0-b133-c5856c4f6161 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:54.755583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 585/1154] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:54.760754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:54.761139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:54.761322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:54.777642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:54.778829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:54.843636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 570/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:54.866731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ec93d-9251-4ee0-b133-c5856c4f6161 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:54.922306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:54.967570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ec93d-9251-4ee0-b133-c5856c4f6161 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.000773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.001554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.001843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.002124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.012142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:13:55.019688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:55.020821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:13:55.021793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 586/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:55.026663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:55.122966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a20d46-a89c-4fcf-b2a6-a71488ea86d9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.136268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:55.156838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.157502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.157791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.158070 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.158824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:55.159138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:55.160550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:55.169250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 571/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:55.174759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:55.271266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b9461-01ad-43ad-a9fd-e64e2f38ff81 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.284697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:55.305145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.306011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.306297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.306573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] PATH_INFO: `/v3/users/2c1f8f887aa640e79aed3e95d0127de6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.307770 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1f8f887aa640e79aed3e95d0127de6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:55.309225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:55.310838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 587/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:13:55 2022] GET /identity/v3/users/2c1f8f887aa640e79aed3e95d0127de6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:55.316658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:55.412693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3276-1bc0-4551-b8dd-e50d6a95ccb4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.444791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.445424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.445710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.445986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.489080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:55.511015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:55.513028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:55.607684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 572/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:55.613443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:55.709614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82670-b854-409a-a911-7836bb7a7649 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.741674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.742517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.742803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.743080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] PATH_INFO: `/v3/projects/3871816c83864bd4ab85314ff1db04a1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.744272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] RBAC: Authorizing `identity:get_project(project_id=3871816c83864bd4ab85314ff1db04a1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:55.745677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb4c9a-e08b-4626-a109-7d2114f55332 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:55.747380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 588/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:13:55 2022] GET /identity/v3/projects/3871816c83864bd4ab85314ff1db04a1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:55.753117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:55.826977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:55.850032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786a74d-33fd-4141-bccf-c31e39348231 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:55.882327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:55.882913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:55.883197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:55.883505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:55.884328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=3871816c83864bd4ab85314ff1db04a1, user.id=2c1f8f887aa640e79aed3e95d0127de6) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:55.884638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:55.886048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313199b-70f5-4612-842b-d57c0d892735 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:55.892290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 573/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:13:55 2022] GET /identity/v3/role_assignments?user.id=2c1f8f887aa640e79aed3e95d0127de6&scope.project.id=3871816c83864bd4ab85314ff1db04a1&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:13:57.903045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c325b9-c8d9-4276-9d7d-0d347112ffa3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:57.903625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c325b9-c8d9-4276-9d7d-0d347112ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:57.903625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c325b9-c8d9-4276-9d7d-0d347112ffa3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:57.904981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 589/1162] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:13:57.910121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:57.910536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:57.910679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:57.927660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:13:57.928852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:13:57.992237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 574/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:13:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:13:58.014187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c325b9-c8d9-4276-9d7d-0d347112ffa3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:58.063684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:58.112329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c325b9-c8d9-4276-9d7d-0d347112ffa3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:58.144379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:58.145014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:58.145297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:58.145571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:58.209310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:58.210737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:58.256816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:58.306207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 590/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:13:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:13:58.312250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:58.377264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:13:58.407159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fe6d6-f656-4620-9b56-593329973e05 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:58.439558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:58.440345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:58.440627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:58.440900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:58.447377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:58.448800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-031ec047-f76e-4de9-a8b8-36319ef0f8c0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:13:58.462522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:13:58.463768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:13:58.465413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 575/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:13:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:13:58.470006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:13:58.565507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebf6a8-2dc5-4b3b-801d-396ebec1fddc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:13:58.598556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:13:58.599207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:13:58.599522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:13:58.599804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:13:58.600525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:13:58.600836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:13:58.602197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f43733-f369-417f-b35d-250c585ccc12 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:13:58.609138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 591/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Wed Jul 6 15:13:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:00.596633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275ff3-47a5-4268-b3fc-2b872bb30aba None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:00.597125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275ff3-47a5-4268-b3fc-2b872bb30aba None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:00.597125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8275ff3-47a5-4268-b3fc-2b872bb30aba None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:00.598471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 576/1167] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:14:00.603500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5607b-ba0c-4f0a-9c13-859fcf347b93 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:00.603891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5607b-ba0c-4f0a-9c13-859fcf347b93 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:00.604067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5607b-ba0c-4f0a-9c13-859fcf347b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:00.620767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-edf5607b-ba0c-4f0a-9c13-859fcf347b93 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:00.621947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-edf5607b-ba0c-4f0a-9c13-859fcf347b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:00.630855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:00.686640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 592/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:00.708918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8275ff3-47a5-4268-b3fc-2b872bb30aba None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:00.807879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8275ff3-47a5-4268-b3fc-2b872bb30aba None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:00.835393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:00.842247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:00.842896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:00.843183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:00.843495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:00.844241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:00.845602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0fb4e7-7c5b-4397-a887-f77cd14ea17f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:00.859484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 577/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Wed Jul 6 15:14:00 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:14:02.863582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c015a-baad-458d-901a-a3f8714267dc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:02.864110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c015a-baad-458d-901a-a3f8714267dc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:02.864220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c015a-baad-458d-901a-a3f8714267dc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:02.865512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 593/1170] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:14:02.870641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:02.871603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:02.871709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:02.888606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:02.889782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:02.954384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 578/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:02.976967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c015a-baad-458d-901a-a3f8714267dc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:03.075011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c015a-baad-458d-901a-a3f8714267dc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:03.107182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:03.108061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:03.108347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:03.108623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] PATH_INFO: `/v3/domains/379af0ed156244ffa76f699ff22decdf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:03.118763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=379af0ed156244ffa76f699ff22decdf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:03.120238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b6e87-633f-4a76-adf0-3a4616816523 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:03.122490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 594/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Jul 6 15:14:02 2022] GET /identity/v3/domains/379af0ed156244ffa76f699ff22decdf => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:03.128049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:03.222789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b21b3d-d474-4133-b5e5-36b0117e6eaf None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:03.242614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:03.257810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:03.258473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:03.258762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:03.259043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:03.259813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:03.261242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c107c26-2034-4189-be79-5dc3a69bd3e3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:03.285379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 579/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:14:03 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:14:05.295038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb7c3-a29b-4f21-a0cc-0e9a9ba1fcfe None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:05.295516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb7c3-a29b-4f21-a0cc-0e9a9ba1fcfe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:05.295630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb7c3-a29b-4f21-a0cc-0e9a9ba1fcfe None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:05.297004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 595/1174] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:14:05.302105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:05.302470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:05.302700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:05.319132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:05.320340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:05.383665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 580/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:05.408135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb7c3-a29b-4f21-a0cc-0e9a9ba1fcfe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:05.411908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:05.505778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb7c3-a29b-4f21-a0cc-0e9a9ba1fcfe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:05.538569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:05.539445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:05.539732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:05.540006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] PATH_INFO: `/v3/domains/379af0ed156244ffa76f699ff22decdf` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:05.541158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] RBAC: Authorizing `identity:get_domain(domain_id=379af0ed156244ffa76f699ff22decdf)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:05.542862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c8bc-92d6-4624-a3bf-498e94a4980e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:05.544542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 596/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Wed Jul 6 15:14:05 2022] GET /identity/v3/domains/379af0ed156244ffa76f699ff22decdf => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:05.550149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:05.645695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262671-bd6b-4191-8a56-3e9a5f243566 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:05.647171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:05.680144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:05.680813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:05.681104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:05.681387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:05.682126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:05.684078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6456467-2e25-476a-8159-256bdda7711c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:05.703090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 581/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:14:05 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:14:07.725068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f781e3-e745-40d6-a2b7-b590e2f179c5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:07.725634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f781e3-e745-40d6-a2b7-b590e2f179c5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:07.725634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f781e3-e745-40d6-a2b7-b590e2f179c5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:07.726978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 597/1178] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:14:07.732528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:07.732947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:07.733103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:07.750147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:07.751368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:07.815852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 582/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:07.838199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f781e3-e745-40d6-a2b7-b590e2f179c5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:07.907107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:07.942472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f781e3-e745-40d6-a2b7-b590e2f179c5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:07.975178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:07.975940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:07.976227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:07.976505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:07.986907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:14:07.994423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:14:07.995605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:14:07.996570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 598/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:14:08.001507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:08.083535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:08.096549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dc1d1-8e67-416a-8198-2bba5423e7e1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:08.129083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:08.129746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:08.130032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:08.130341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:08.131091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:14:08.131430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:08.132801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:08.140669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 583/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:08.146285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:08.244083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4068cd-a172-4071-a46d-daa0913bd5b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:08.272294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:08.278125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:08.278988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:08.279295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:08.279571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] PATH_INFO: `/v3/users/aa8aae028dd54fbe87eaf28b8178419d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:08.292134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] RBAC: Authorizing `identity:get_user(user_id=aa8aae028dd54fbe87eaf28b8178419d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:08.293852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:08.295511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 599/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:14:08 2022] GET /identity/v3/users/aa8aae028dd54fbe87eaf28b8178419d => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:08.301239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:08.396053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58328a02-b2fe-459c-9e04-407c74d2c2f5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:08.428191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:08.428829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:08.429113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:08.429390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:08.447194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:08.494744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:08.496169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:08.589395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 584/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:14:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:14:08.595793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:08.691578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ea7ef-96ee-43c3-b9f6-c90ce24ad414 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:08.724281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:08.725130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:08.725661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:08.725661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] PATH_INFO: `/v3/projects/b5f9a41483c24728a63e7db55abcc85a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:08.726832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] RBAC: Authorizing `identity:get_project(project_id=b5f9a41483c24728a63e7db55abcc85a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:08.728309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d8705-5669-419f-bcf4-86af08f202ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:08.729966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 600/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:14:08 2022] GET /identity/v3/projects/b5f9a41483c24728a63e7db55abcc85a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:08.735766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:08.783161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:08.830522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd366f0-01c1-44c2-8cfb-6f61e634357d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:08.862057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:08.862695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:08.862976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:08.863284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:08.864110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=b5f9a41483c24728a63e7db55abcc85a, user.id=aa8aae028dd54fbe87eaf28b8178419d) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:14:08.864423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:08.865833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f4ee7-9249-41ba-8712-4217dc30489c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:08.871553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 585/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:14:08 2022] GET /identity/v3/role_assignments?user.id=aa8aae028dd54fbe87eaf28b8178419d&scope.project.id=b5f9a41483c24728a63e7db55abcc85a&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:10.826124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93313b-4c60-485f-8f86-09611de7bcdd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:10.826666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93313b-4c60-485f-8f86-09611de7bcdd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:10.826666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93313b-4c60-485f-8f86-09611de7bcdd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:10.828080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 601/1186] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:14:10.833443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540747af-e9a2-4938-8d89-280256842015 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:10.833861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540747af-e9a2-4938-8d89-280256842015 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:10.834001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540747af-e9a2-4938-8d89-280256842015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:10.850773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-540747af-e9a2-4938-8d89-280256842015 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:10.851990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-540747af-e9a2-4938-8d89-280256842015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:10.915864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 586/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:10.937665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93313b-4c60-485f-8f86-09611de7bcdd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:10.994410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:11.035474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93313b-4c60-485f-8f86-09611de7bcdd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:11.067379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:11.068153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:11.068436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:11.068712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:11.078968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:14:11.086408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:14:11.087369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:14:11.088426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 602/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:14:11.093958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540747af-e9a2-4938-8d89-280256842015 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:11.135836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:11.188005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540747af-e9a2-4938-8d89-280256842015 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:11.221138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:11.221801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:11.222088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:11.222364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:11.223112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:14:11.223449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:11.224815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:11.232651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 587/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:11.238148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:11.335524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a79a2-484e-4564-a092-f6a5b913073b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:11.351570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:11.368008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:11.368880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:11.369167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:11.369448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] PATH_INFO: `/v3/users/aa8aae028dd54fbe87eaf28b8178419d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:11.371147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] RBAC: Authorizing `identity:get_user(user_id=aa8aae028dd54fbe87eaf28b8178419d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:11.372605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:11.374233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 603/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:14:11 2022] GET /identity/v3/users/aa8aae028dd54fbe87eaf28b8178419d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:11.379988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:11.475661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e5a96-d0d0-4ef7-baaf-526542c2f321 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:11.482300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:11.508629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:11.509473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:11.509758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:11.510033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] PATH_INFO: `/v3/projects/b5f9a41483c24728a63e7db55abcc85a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:11.511307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] RBAC: Authorizing `identity:get_project(project_id=b5f9a41483c24728a63e7db55abcc85a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:11.513168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d509d68-c80f-49eb-9c44-de3752b431e9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:11.514805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 588/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:14:11 2022] GET /identity/v3/projects/b5f9a41483c24728a63e7db55abcc85a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:11.521095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:11.617039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a15c74-7b67-480f-8c57-f81137adb1b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:11.649869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:11.650741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:11.651026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:11.651333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] PATH_INFO: `/v3/projects/b5f9a41483c24728a63e7db55abcc85a/users/aa8aae028dd54fbe87eaf28b8178419d/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:11.653294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5f9a41483c24728a63e7db55abcc85a, user_id=aa8aae028dd54fbe87eaf28b8178419d, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:11.654706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d3cd5-2352-47d3-9a0f-e20c0c432753 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:11.666694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 604/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:14:11 2022] PUT /identity/v3/projects/b5f9a41483c24728a63e7db55abcc85a/users/aa8aae028dd54fbe87eaf28b8178419d/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:14:13.629682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44e80-17d1-4222-9475-5a08a3d09e85 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:13.630174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44e80-17d1-4222-9475-5a08a3d09e85 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:13.630174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d44e80-17d1-4222-9475-5a08a3d09e85 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:13.631544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 589/1193] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:14:13.636535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:13.636954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:13.637099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:13.653940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:14:13.655128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:14:13.696510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:13.720949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 605/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:14:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:14:13.744060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44e80-17d1-4222-9475-5a08a3d09e85 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:13.842126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d44e80-17d1-4222-9475-5a08a3d09e85 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:13.874823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:13.875635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:13.875922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:13.876200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:13.886051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:14:13.894047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:14:13.895256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:14:13.896485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 590/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:14:13.900946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:13.996826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60e6de-3bba-4fe7-a9f3-f0563bb6b14d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:14.030041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:14.030705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:14.030995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:14.031298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:14.032051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:14:14.032362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:14.033724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:14.041567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 606/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:14.047901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:14.081437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:14.144661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7164-7b95-4b85-b1aa-b3cc8df58717 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:14.176761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:14.177614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:14.177899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:14.178177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] PATH_INFO: `/v3/users/aa8aae028dd54fbe87eaf28b8178419d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:14.179373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] RBAC: Authorizing `identity:get_user(user_id=aa8aae028dd54fbe87eaf28b8178419d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:14.180799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:14.182379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 591/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Wed Jul 6 15:14:14 2022] GET /identity/v3/users/aa8aae028dd54fbe87eaf28b8178419d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:14.188605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:14.226701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:14.285915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6be850-19e8-4851-8aba-5ff5cef905e6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:14.318878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:14.319552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:14.319840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:14.320126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:14.384122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:14.385554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:14.424887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:14.482545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 607/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:14:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:14:14.488368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:14.585920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d36f-a170-477a-adc1-aeb5b61fa4ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:14.587438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:14.620051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:14.620910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:14.621198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:14.621476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] PATH_INFO: `/v3/projects/b5f9a41483c24728a63e7db55abcc85a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:14.622652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] RBAC: Authorizing `identity:get_project(project_id=b5f9a41483c24728a63e7db55abcc85a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:14.624113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10379a66-6368-41f6-9714-478dabc322bc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:14.625777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 592/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Wed Jul 6 15:14:14 2022] GET /identity/v3/projects/b5f9a41483c24728a63e7db55abcc85a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:14.631750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:14:14.728153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460af09-5603-4e26-b0a6-b0a1a9c99407 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:14:14.760061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:14:14.760705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:14:14.760993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:14:14.761799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:14:14.762637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=b5f9a41483c24728a63e7db55abcc85a, user.id=aa8aae028dd54fbe87eaf28b8178419d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:14:14.762955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:14:14.764408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b55b-9918-4c27-bb8e-67af6dfa6571 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:14:14.770074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 608/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:14:14 2022] GET /identity/v3/role_assignments?user.id=aa8aae028dd54fbe87eaf28b8178419d&scope.project.id=b5f9a41483c24728a63e7db55abcc85a&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:14:24.616119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=15 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:14:24.767531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:09.703707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9302-9f0c-464a-b237-827a48bdbaab None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:09.704304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9302-9f0c-464a-b237-827a48bdbaab None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:09.704405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f9302-9f0c-464a-b237-827a48bdbaab None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:09.705933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 593/1201] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:09.711559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:09.712107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:09.712194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:09.731071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:09.732326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:09.816219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 609/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:15:09.841691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f9302-9f0c-464a-b237-827a48bdbaab None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:09.970656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f9302-9f0c-464a-b237-827a48bdbaab None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:10.012622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:10.013453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:10.013774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:10.014105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:10.027829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:10.029554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:10.052177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:10.053221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:10.054709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:10.054709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:10.054709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:10.054709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:10.054709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 594/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:10.060832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:10.190003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0403ab-5846-4e66-8c0d-57ee2baa377e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:10.222909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:10.235326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:10.235326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:10.235326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:10.235326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:10.235766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:10.235766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:10.235766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce6002-0dc7-4000-9815-c1b78383ecb0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:10.253697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 610/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:10.255395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:10.323795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:10.390644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042a773-fc77-4345-9816-b9b8ef0e0465 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:10.432762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:10.433572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:10.433908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:10.434267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:10.435099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:10.436616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77693632-e30b-44c4-9199-7fe946ec655e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:10.471503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 595/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Wed Jul 6 15:15:10 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:15:12.682644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832591e-b75f-4e04-8fbf-c0b3408e87c1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:12.683132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832591e-b75f-4e04-8fbf-c0b3408e87c1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:12.683132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832591e-b75f-4e04-8fbf-c0b3408e87c1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:12.684524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 611/1206] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:12.689844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:12.690285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:12.690483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:12.709408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:12.710531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:12.787283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 596/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:12.810948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832591e-b75f-4e04-8fbf-c0b3408e87c1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:12.921803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832591e-b75f-4e04-8fbf-c0b3408e87c1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:12.961043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:12.962368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:12.962656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:12.962932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:12.976302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:12.985703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:12.987065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:12.987977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 612/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:15:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:15:12.995374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:13.041523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:13.112396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae09cf-0db0-4bcb-b046-93fcb601a6a1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:13.157021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:13.157701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:13.157998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:13.158271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:13.159168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:13.159666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:13.161255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:13.172274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 597/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:15:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:13.178355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:13.243475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:13.300057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac411f8-1826-4dab-942b-f01f05344c65 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:13.343039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:13.343766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:13.344051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:13.344325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:13.355723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:13.357170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:13.374682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:13.376048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:13.376963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:13.376963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:13.376963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:13.376963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:13.376963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 613/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:15:13.381951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:13.491446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:13.494474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db91e-0d1b-4624-bb07-53544a2318ee None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:13.531726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:13.532369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:13.532653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:13.532934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:13.533655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:13.533967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:13.535362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:13.559249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 598/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:13.565427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:13.676906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f552ace-51f0-4b90-946f-1929f1ff3515 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:13.678797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:13.719944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:13.722927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:13.722927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:13.722927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:13.728838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:13.730450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:13.748692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:13.749751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:13.755806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 614/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:13.756434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:13.887832 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866cd9b-1075-481a-b84f-2946767b9e1a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:13.926361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:13.927045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:13.927368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:13.927641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:13.928522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:13.928840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:13.930256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:13.949698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 599/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:13.956599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:14.077905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009acc7-9c48-433a-8013-06a914f6565f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:14.116639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:14.117971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:14.117971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:14.118705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:14.154355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:14.202166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:14.203773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:14.324841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 615/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:15:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 15:15:14.331510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:14.353040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:14.447273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba6cdd-001b-4433-9b4d-545540799df3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:14.483703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:14.484576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:14.484904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:14.485122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:14.497241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:14.498903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:14.519556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:14.520710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:14.521615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:14.521615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:14.521615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:14.521615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:14.521615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 600/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:14.527533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:14.597785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:14.650304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15b27c-d02e-4d36-90b4-7125e59807a4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:14.690071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:14.690767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:14.691056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:14.691382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:14.692139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:14.692454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:14.693854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:14.707795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 616/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:14.714163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:14.790791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:14.829750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e3335-0c1c-494c-851f-4e118eaf1dc6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:14.869192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:14.870032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:14.870319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:14.870696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:14.879622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:14.881327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:14.897089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:14.898280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:14.900132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 601/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:14.905494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:15.017214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00026e87-a8ad-488a-9196-6589206c086c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:15.048698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:15.067751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:15.068847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:15.069274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:15.069792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:15.070827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:15.071998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:15.074190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e6a02a-cc0f-41da-b1c7-9824e745aaf4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:15.090875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 617/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:15.099767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:15.215325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d938a6-23cf-4f51-a183-012ad216e2a9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:15.248559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:15.258289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:15.259630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:15.259977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:15.260314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:15.261302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=319c09d5d4db4a18bf1191375252c541) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:15.261686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:15.263236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea86c0-bc60-44e0-a569-6cfbbf1c5642 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:15.279249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 602/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:15:15 2022] GET /identity/v3/role_assignments?user.id=319c09d5d4db4a18bf1191375252c541&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:17.445825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5016e0b-7636-4724-aed0-47779c22265e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:17.446424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5016e0b-7636-4724-aed0-47779c22265e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:17.446424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5016e0b-7636-4724-aed0-47779c22265e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:17.447815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 618/1220] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:17.453111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:17.453574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:17.453800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:17.472286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:17.473640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:17.545700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 603/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:17.569143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5016e0b-7636-4724-aed0-47779c22265e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:17.680057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5016e0b-7636-4724-aed0-47779c22265e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:17.716958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:17.717760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:17.718050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:17.718330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:17.729717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:17.739691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:17.740665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:17.741514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 619/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:17.745873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:17.846243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01213c79-a026-4932-8f79-257b9f3a0a7a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:17.869239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:17.879933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:17.880610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:17.880893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:17.881168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:17.881918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:17.882226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:17.884152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:17.892821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 604/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:15:17.898454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:17.944219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:18.011128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba26ba0-2a98-4df7-b9d0-c71425b85494 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.051556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.052368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.052632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.052909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.064529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:18.066050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:18.083371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:18.084435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:18.085456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:18.085456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:18.085456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:18.085456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:18.085456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 620/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:18.090998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:18.199140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f837449-87b0-421d-b7c7-e7eba4ec5973 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.235516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.236210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.236533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.236904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.237641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:18.238007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:18.239454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:18.257035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 605/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:18.264023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:18.370920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:18.378682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b452bc0-0b19-409e-98db-c4ba9e5ef748 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.416148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.416951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.417233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.417503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.425621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:18.427140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:18.443564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:18.444754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:18.445730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 621/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:18.451326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:18.506801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:18.568940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070acdb-0454-460e-bcc8-8c3935017369 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.606041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.606749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.607032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.607338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.608188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:18.608495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:18.609941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:18.629360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 606/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:18.635196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:18.747983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab3c28-300d-43e8-9c13-01d59e19ae2d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.784673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.785833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.786185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.786467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.799005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:18.801211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:18.820026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:18.821241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:18.822296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:18.822296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:18.822296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:18.822296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:18.822296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 622/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:18.828289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:18.952350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71836340-5869-4c4f-8338-e943ab702b4c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:18.955109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:18.995081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:18.996077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:18.996456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:18.996838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:18.997874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:18.998385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:19.000284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:19.013494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 607/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:15:19.020096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:19.031021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:19.130284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2ee3-2bba-47c7-9137-2c80da088631 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:19.167685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:19.168681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:19.169112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:19.169555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:19.177375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:19.178825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:19.194793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:19.195944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:19.197171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 623/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:19.202007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:19.311589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce27d0-3f46-43e6-9011-4beb08d57e62 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:19.346597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:19.347294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:19.347585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:19.347865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:19.348669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:19.348987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:19.350423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5127f2-96b3-4966-8b88-81eb2efb5d93 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:19.363186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 608/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:19.369799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:19.438686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:19.478533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e4fe-9f36-4d2c-8972-d20df7e9e387 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:19.515293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:19.516120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:19.516410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:19.516681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/319c09d5d4db4a18bf1191375252c541/roles/7d18029c5f85482aad78e7a7dec28b60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:19.533171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=319c09d5d4db4a18bf1191375252c541, role_id=7d18029c5f85482aad78e7a7dec28b60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:19.534757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de8f2e-2246-4d59-a1ba-0a6d22048a4c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:19.547244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 624/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:15:19 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/319c09d5d4db4a18bf1191375252c541/roles/7d18029c5f85482aad78e7a7dec28b60 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:15:21.612594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b10-1561-44ac-ac82-14b2c296df70 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:21.613115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b10-1561-44ac-ac82-14b2c296df70 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:21.613115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c835b10-1561-44ac-ac82-14b2c296df70 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:21.614435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 609/1233] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:21.620449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:21.620959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:21.621086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:21.641869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:21.643369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:21.717150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 625/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:21.740536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c835b10-1561-44ac-ac82-14b2c296df70 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:21.767174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:21.855629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c835b10-1561-44ac-ac82-14b2c296df70 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:21.895469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:21.896477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:21.896794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:21.897078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:21.908138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:21.916081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:21.917172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:21.918180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 610/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Wed Jul 6 15:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:21.923441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:21.972186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:22.037682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e057a-98e1-4ee6-a29b-d12f750ca3bf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.075405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.076145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.076488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.076867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:22.077671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:22.077999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:22.079583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:22.088379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 626/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Wed Jul 6 15:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:22.094111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:22.190119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:22.206888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f679b-e267-4d82-9f33-e7d5bc39c908 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.246394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.247606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.247894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.248173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:22.260846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:22.262469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:22.282344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:22.283451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:22.284451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:22.284451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:22.284451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:22.284451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:22.284451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 611/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:15:22.289965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:22.406228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efcf42-56b3-4d6a-b6e9-45ac0ed6196f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.415995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:22.442789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.443434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.443755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.444028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:22.444789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:22.445098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:22.446557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:22.464119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 627/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:22.465650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:22.572146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfc585-b716-4d41-9e8d-e6f9eb04389b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.609186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.610090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.610373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.610649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:22.613447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:22.619423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:22.621061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:22.635758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:22.636848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:22.637874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:22.638449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 612/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:22.643995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:22.757462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85572b6-c739-4218-ae3f-b5de3b00197a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.799485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.800166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.800490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.800869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:22.801938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:22.802251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:22.804039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:22.822022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 628/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:22.828532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:22.932582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99511ecb-2ea7-4bd2-b893-4887eda1089b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:22.971385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:22.972909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:22.973199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:22.973487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:23.054994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:23.064680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:23.066885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:23.177993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 613/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:15:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:15:23.184895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:23.217279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:23.301615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abc2ac-4e6f-4b39-af9e-f41941ccf056 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:23.340514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:23.341315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:23.341596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:23.341873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:23.352886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:23.354335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:23.374230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:23.375301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:23.376230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:23.376230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:23.376230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:23.376230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:23.376230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 629/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:23.381144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:23.479056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:23.486021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731ee0f-1c72-4fba-a991-4c328f0df7a9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:23.522916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:23.523617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:23.523902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:23.524179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:23.524909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:23.525224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:23.526780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:23.538539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 614/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:23.544425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:23.630174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:23.655030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7afb74-03f8-4057-92e7-6bf827e87eba None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:23.692247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:23.693077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:23.693946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:23.694271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:23.701697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:23.703115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:23.716892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:23.717926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:23.718915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 630/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:23.724010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:23.833554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540a0de-6d77-426c-b6f1-a89997ee55f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:23.874364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:23.875034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:23.875349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:23.875629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:23.876418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:23.876726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:23.878152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5e2e-6559-4d95-af9b-0a2697af0dcd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:23.890623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 615/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:23.896721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:24.006292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f88a2-9d4c-47ae-9f02-80a8f0a7ed2f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:24.049850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:24.050621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:24.051040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:24.051382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:24.052242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] RBAC: Adding query filter params (role.id=7d18029c5f85482aad78e7a7dec28b60, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=319c09d5d4db4a18bf1191375252c541) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:24.052596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:24.054147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3475f-cc5f-4c08-9dc1-c136585b2e63 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:24.061537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 631/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:15:23 2022] GET /identity/v3/role_assignments?user.id=319c09d5d4db4a18bf1191375252c541&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=7d18029c5f85482aad78e7a7dec28b60 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:26.514188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0b13e-338f-4537-bce7-677c352bae5d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:26.514777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0b13e-338f-4537-bce7-677c352bae5d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:26.514777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0b13e-338f-4537-bce7-677c352bae5d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:26.516278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 616/1247] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:26.521502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:26.521938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:26.522047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:26.527879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:26.541702 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:26.543006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:26.618453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 632/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:26.641974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0b13e-338f-4537-bce7-677c352bae5d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:26.661432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:26.761694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0b13e-338f-4537-bce7-677c352bae5d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:26.799585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:26.801001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:26.801617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:26.802131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:26.816749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:26.827008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:26.828191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:26.829245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 617/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:26.834925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:26.948992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f1d2-3b56-4776-bb8f-36ea8a203fb4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:26.989808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:26.996297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:26.997148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:26.997432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:26.997711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:26.998481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:26.998783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.000255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:27.011278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 633/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:27.018051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:27.105072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:27.130235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6c6f87-2d22-417a-8b26-7a92a2f7b70a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:27.164998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:27.166270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:27.166600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:27.166938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:27.179197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.181242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:27.200161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:27.201241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:27.202169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:27.202169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:27.202169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:27.202169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:27.207858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 618/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:27.209099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:27.320598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfc86c-ed5e-4acc-8d2b-14d2873858cd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:27.357771 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:27.358460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:27.358694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:27.358975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:27.359830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:27.360206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.361553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:27.374898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 634/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:27.380615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:27.492162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a3ac-0271-4eb6-875a-6056ce6f4223 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:27.521187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:27.530297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:27.531328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:27.531712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:27.531974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:27.541351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.543003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:27.559069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:27.560184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:27.561134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 619/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:27.566810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:27.627980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:27.675517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423422b-3a80-424f-a831-05d782c95fa0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:27.717854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:27.718530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:27.718814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:27.719090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:27.719978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:27.720293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.721722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:27.740143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 635/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:27.746556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:27.849138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e540c83-fe17-4818-be28-5ed9e19af8f9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:27.890368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:27.891305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:27.891589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:27.891864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:27.948329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:27.963202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:27.964688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.077233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 620/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:15:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:15:28.082975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:28.184636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534e5e-7b0c-4aeb-ab5e-55a99f88640b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:28.198828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:28.220923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:28.221714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:28.221995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:28.222278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:28.232428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:28.233885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:28.252350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:28.253427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:28.254685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:28.254685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:28.254685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:28.254685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:28.254685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 636/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:28.260037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:28.353479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:28.366342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3da66-fd91-4cfe-8d8f-7eeb9f0a67f4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:28.407605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:28.408272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:28.408597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:28.408917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:28.409686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:28.409998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:28.411433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.423822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 621/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:28.429816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:28.537521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc70ec0-e306-426f-9840-bbcd2fa3dd80 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:28.576190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:28.577016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:28.577300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:28.577578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:28.585454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:28.586989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:28.601745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:28.602937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:28.604078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 637/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:28.609086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:28.716584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a909165-bf27-4059-b51f-9b2c7f566d8c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:28.751983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:28.752661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:28.752947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:28.753228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:28.754035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:28.754352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:28.755856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1820151-96a9-4aed-a488-c13466c8e7be None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.774934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 622/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:28.776243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:28.803540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:28.881490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7836b-2352-4f49-8402-1c2a87059b57 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:28.917124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:28.917800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:28.918090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:28.918361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:28.919211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=319c09d5d4db4a18bf1191375252c541) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:28.919551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:28.920993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b4b0-9efa-477a-ada7-dd7a32c08eb5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:28.939266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 638/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:15:28 2022] GET /identity/v3/role_assignments?user.id=319c09d5d4db4a18bf1191375252c541&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:30.923595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a0453-1399-46aa-b17c-33bc7b91d0b9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:30.924184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a0453-1399-46aa-b17c-33bc7b91d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:30.924184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a0453-1399-46aa-b17c-33bc7b91d0b9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:30.925515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 623/1261] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:30.930746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:30.931131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:30.931332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:30.947617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:30.948795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:31.023268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 639/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:31.049030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a0453-1399-46aa-b17c-33bc7b91d0b9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.060599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.147759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a0453-1399-46aa-b17c-33bc7b91d0b9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.180690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.181416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.181702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.181979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.191591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:31.199608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:31.200708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:31.201823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 624/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:15:31.206701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.224722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.303955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf355b8-81a3-4aaf-a0f3-0afdd53f7403 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.336720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.337377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.337659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.337932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.338674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:31.338979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:31.340353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:31.348094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 640/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:31.353704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.419000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.452249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5ca6-f92d-4477-b184-d03649a24de8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.484547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.485326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.485613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.485890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.496724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:31.498161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:31.515165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:31.516209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:31.517505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:31.517505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:31.517505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:31.517505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:31.517505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 625/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:31.521987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.600461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.619621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f999a-d15d-4e6a-a09b-eaccd9b96ba0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.651680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.652341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.652628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.652907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.653625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:31.653938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:31.655320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:31.666872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 641/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:31.672662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.769751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd86a-0ca8-4da1-9d7c-57c72da19458 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.788401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.802622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.803441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.803733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.804014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.811743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:31.813179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:31.827662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:31.828753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:31.830066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 626/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:31.834565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:31.931371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300a799-15bd-49ee-9689-586f6c19f7e0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:31.963599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:31.964254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:31.964539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:31.964817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:31.965647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:31.965958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:31.967455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:31.977344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:31.987191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 642/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:31.993974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:32.095383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b63ad-d4b6-476d-a876-48f593d43662 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:32.127879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:32.128670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:32.128957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:32.129237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:32.139420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:32.140860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:32.157264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:32.158193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:32.159115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:32.159115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:32.159115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:32.159115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:32.159115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 627/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:32.164610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:32.261281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b4137-687e-495f-bd40-2cd405c51498 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:32.294838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:32.295532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:32.295816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:32.296094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:32.296812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:32.297120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:32.298491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:32.309205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 643/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:32.314741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:32.347570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:32.419577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb54af7-c6ce-45f4-a92c-7dc395c65e86 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:32.451136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:32.451943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:32.452230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:32.452509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:32.458933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:32.460393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:32.474725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:32.475833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:32.477798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 628/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:32.482200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:32.537273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:32.577140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b552a0-a945-4465-993e-4440ef5f3159 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:32.609313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:32.609905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:32.610189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:32.610462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:32.611261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:32.611573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:32.612973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc62fa-c698-4222-868a-46446167a41f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:32.624339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 644/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:15:32.629997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:32.718711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:32.727688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8752bd-87b3-47ec-91f9-afd180dc10cf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:32.760355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:32.761231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:32.761515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:32.761794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] PATH_INFO: `/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/319c09d5d4db4a18bf1191375252c541/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:32.763758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da64de84a214a5bbc0d2c0105cc892a, user_id=319c09d5d4db4a18bf1191375252c541, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:32.765182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c554e2-a2fa-405a-bc29-056e6af1c6f1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:32.777709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 629/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Wed Jul 6 15:15:32 2022] PUT /identity/v3/projects/0da64de84a214a5bbc0d2c0105cc892a/users/319c09d5d4db4a18bf1191375252c541/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:15:34.765673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed59a-bac2-4cf9-8dde-8af85665c007 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:34.766187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed59a-bac2-4cf9-8dde-8af85665c007 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:34.766280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ed59a-bac2-4cf9-8dde-8af85665c007 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:34.767629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 645/1274] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:34.772545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:34.772916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:34.773109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:34.791298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:34.792463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:34.862388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 630/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:34.890793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed59a-bac2-4cf9-8dde-8af85665c007 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:34.991621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ed59a-bac2-4cf9-8dde-8af85665c007 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.009216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:35.026181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.026912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.027198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.027506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.037322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 15:15:35.045199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:35.046259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:35.047149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 646/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Wed Jul 6 15:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:35.052250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.128220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:35.148134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72671694-e9e6-4079-a359-9bd3a68c6f56 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.180588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.181243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.181528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.181807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.182555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:35.182861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:35.184242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:35.191944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 631/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:35.197902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.294818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5869d4-fcf1-4471-896d-7b02e5a46c89 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.327075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.327884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.328169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.328443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.338600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:35.340044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:35.356413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:35.357432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:35.358383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:35.358383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:35.358383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:35.358383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:35.358383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 647/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:35.363163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.458310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82333-6aa0-412d-997f-15257ab945a1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.489884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.490540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.490825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.491103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.491852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:35.492657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:35.494021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:35.505340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 632/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:35.511298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.540653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:35.608017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf6085-70a0-458d-a29e-4fe7a7ee30f3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.640767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.641549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.641833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.642109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.650063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:35.651516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 15:15:35.666315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:35.667462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:35.668476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 648/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:35.673046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.676751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:35.768612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47501d65-d49a-4e7a-b391-fcf8b834e79f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.801223 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.801885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.802171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.802448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:35.803312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:35.803630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:35.805661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:35.821842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 633/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:35.827475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:35.913793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:35.924312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69595b58-4ccb-44a4-8889-8671b0002021 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:35.956522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:35.957161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:35.957450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:35.957730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.032516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.033966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.118875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:36.129152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 649/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Wed Jul 6 15:15:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 15:15:36.134836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:36.194484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:36.231764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478272c1-d85b-4a2c-b8ec-314e73679c1d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:36.264106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:36.264894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:36.265181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:36.265461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.275613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.277045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 15:15:36.295535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:36.296572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:36.297682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 15:15:36.297682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 15:15:36.297682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 15:15:36.297682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:36.297682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 634/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:36.302251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:36.396064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003cad9-8c69-4652-9375-5fd39a814852 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:36.429210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:36.429887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:36.430169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:36.430444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.431161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:36.431497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.432839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.443538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 650/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Wed Jul 6 15:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:36.449474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:36.545042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c215-7c6f-4856-baf9-71dc04c6d085 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:36.558384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:36.578143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:36.578951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:36.579242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:36.579542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.586601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.588066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 15:15:36.601460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:36.602500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jul 06 15:15:36.603634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 635/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Wed Jul 6 15:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:36.608275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:36.657081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:36.705011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034bdc0-1f55-41c7-8794-dc09c7ec39ef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:36.737002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:36.737656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:36.737942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:36.738220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.739006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:36.739359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.740781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9812fb5-b4b9-4153-9d17-e7c8da8318c7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.752291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 651/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Wed Jul 6 15:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:36.758627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:36.861192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a587d-1fa0-4a52-b1e8-439f92f7a243 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:36.896390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:36.897036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:36.897318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:36.897591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:36.898932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] RBAC: Adding query filter params (role.id=84f17f17b9694564b251da9fa908f805, scope.project.id=0da64de84a214a5bbc0d2c0105cc892a, user.id=319c09d5d4db4a18bf1191375252c541) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:36.899271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:36.900687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13d7e6-0c1c-46b0-86b9-50c2dc39064a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:36.906368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 636/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Wed Jul 6 15:15:36 2022] GET /identity/v3/role_assignments?user.id=319c09d5d4db4a18bf1191375252c541&scope.project.id=0da64de84a214a5bbc0d2c0105cc892a&role.id=84f17f17b9694564b251da9fa908f805 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:38.910010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42995-7728-4361-b053-92cf9927dabf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:38.910573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42995-7728-4361-b053-92cf9927dabf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:38.910573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42995-7728-4361-b053-92cf9927dabf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:38.911999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 652/1288] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:38.917493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:38.917908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:38.918051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:38.934257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:38.935465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:38.964683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:39.007261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 637/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 15:15:39.032676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42995-7728-4361-b053-92cf9927dabf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:39.131470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42995-7728-4361-b053-92cf9927dabf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:39.142797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:39.165155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:39.165882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:39.166166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:39.166441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:39.167083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:39.169063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:39.181017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:39.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:39.183119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 653/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:15:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:39.188175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:39.285570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bb769-c41e-48e7-b008-8bcdba2ae27d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:39.318148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:39.318814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:39.319099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:39.319407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:39.320757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:39.321068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:39.322435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f0168-818c-42ae-abc5-86427cd13892 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:39.327106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 638/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 6 15:15:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:39.332759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:39.428241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc321b1-d3c6-474e-a4b2-0c716a62b0b4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:39.459748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:39.460389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:39.460698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:39.460969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:39.461664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:39.461971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:39.463335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244efb0-fee1-4d3c-96eb-27a1dee1fe0e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:39.467575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 654/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 6 15:15:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:41.438919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0058d7-70ea-4a8d-b879-44d961fbab1e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:41.439477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0058d7-70ea-4a8d-b879-44d961fbab1e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:41.439586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0058d7-70ea-4a8d-b879-44d961fbab1e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:41.440862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 639/1293] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:41.445973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599968b-4002-4716-9b47-47d1413d1cf7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:41.446367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599968b-4002-4716-9b47-47d1413d1cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:41.446531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599968b-4002-4716-9b47-47d1413d1cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:41.463091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8599968b-4002-4716-9b47-47d1413d1cf7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:41.464290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8599968b-4002-4716-9b47-47d1413d1cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:41.492882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:41.528479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 655/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:41.551094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0058d7-70ea-4a8d-b879-44d961fbab1e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:41.581374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:41.649915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0058d7-70ea-4a8d-b879-44d961fbab1e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:41.682954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:41.683637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:41.683929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:41.684210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:41.684928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:41.686292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a852-fadf-4655-bc74-5969a009721a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:41.699161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 640/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Wed Jul 6 15:15:41 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:15:43.697713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49201-b536-47fb-adb1-ae4727e51717 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:43.698239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49201-b536-47fb-adb1-ae4727e51717 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:43.698239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c49201-b536-47fb-adb1-ae4727e51717 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:43.699625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 656/1296] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:43.704719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:43.705132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:43.705283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:43.721500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:43.722667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:43.792698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 641/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:43.815272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c49201-b536-47fb-adb1-ae4727e51717 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:43.911946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c49201-b536-47fb-adb1-ae4727e51717 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:43.944616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:43.945421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:43.945706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:43.945980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:43.946625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:43.948046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:43.959419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:43.960668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:43.961809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 657/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:15:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 15:15:43.966288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:43.986634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:44.076504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625ff12-cc1f-4986-b20d-fb6198afcdce None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:44.108804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:44.109466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:44.109751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:44.110032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:44.110739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:44.111048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:44.112442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:44.118023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 642/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 6 15:15:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:44.123965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:44.153772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:44.220981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd261ee1-47f7-404a-825f-ab6bc14f3a44 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:44.253752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:44.254406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:44.254692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:44.254972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:44.255796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] RBAC: Adding query filter params (interface=public, service_id=ad5ebb34840341b0b59d37b33881a51a, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:44.256111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:44.257473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1b96e-01e8-4b2a-af23-51516659d514 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:44.262366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 658/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Wed Jul 6 15:15:44 2022] GET /identity/v3/endpoints?service_id=ad5ebb34840341b0b59d37b33881a51a&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:44.267862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:44.346614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:44.364426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a30913-c095-4add-8c10-9cb5826133e1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:44.396253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:44.396910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:44.397197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:44.397477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:44.398935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:44.400335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045df82-584c-40ac-84a5-ddca5d232a4e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:44.406897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 643/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Wed Jul 6 15:15:44 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:15:46.386756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9b98-5898-4479-8ded-d0a71c30762e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:46.387340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9b98-5898-4479-8ded-d0a71c30762e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:46.387340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9b98-5898-4479-8ded-d0a71c30762e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:46.388703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 659/1302] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:15:46.393958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:46.394353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:46.394529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:46.411470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:15:46.412655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:15:46.476680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 644/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:15:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 15:15:46.498435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9b98-5898-4479-8ded-d0a71c30762e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:46.590861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:46.597822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9b98-5898-4479-8ded-d0a71c30762e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:46.629903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:46.630682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:46.630964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:46.631261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:46.631907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:46.633859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 15:15:46.645338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 15:15:46.646484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jul 06 15:15:46.647404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 660/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Wed Jul 6 15:15:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 15:15:46.652282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:46.726125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:46.748794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e529a8-b586-4764-9e05-5254f53b9e4d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:46.781328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:46.782004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:46.782291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:46.782572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:46.783308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:15:46.783621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:46.784979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c2158-7652-452e-90e1-8d0e2c7c66a7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:46.790292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 645/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Wed Jul 6 15:15:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:15:46.795790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:15:46.892960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0ed0e-3b69-43f9-821d-2d7b4be5a6e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:15:46.925050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:15:46.925688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:15:46.925973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:15:46.926251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:15:46.926988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:15:46.928375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca5720-59ec-4c48-b755-7bf4fa18351a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:15:46.946975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 661/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Wed Jul 6 15:15:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:15:56.787013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=30 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:15:56.942627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:49.178871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53515e-6eb0-4a58-abec-afdec76d5d3f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:49.179438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53515e-6eb0-4a58-abec-afdec76d5d3f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:49.179438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53515e-6eb0-4a58-abec-afdec76d5d3f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:49.185984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2b80-58d5-4966-9e53-d47fff876528 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:49.186421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2b80-58d5-4966-9e53-d47fff876528 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:49.186656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2b80-58d5-4966-9e53-d47fff876528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:49.189135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 646/1307] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:16:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:16:49.280606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f3aa2b80-58d5-4966-9e53-d47fff876528 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:16:49.281831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3aa2b80-58d5-4966-9e53-d47fff876528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:49.559266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 662/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:16:49.598703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:49.599000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:49.599503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:49.601083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 647/1309] 10.222.0.46 () {54 vars in 964 bytes} [Wed Jul 6 15:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:16:49.607011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c7bb-8c72-4491-b91d-1f1ebec1caed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:49.607333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c7bb-8c72-4491-b91d-1f1ebec1caed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:49.607618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861c7bb-8c72-4491-b91d-1f1ebec1caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:49.684045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5861c7bb-8c72-4491-b91d-1f1ebec1caed None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:16:49.685170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5861c7bb-8c72-4491-b91d-1f1ebec1caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:49.707751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:49.769235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 663/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Jul 6 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:16:49.778544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:16:49.780378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:49.893269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f22ba-a682-40c8-aa5d-03096c43a65d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:16:49.970324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:16:49.970981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:49.971471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:49.971754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:49.973953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:50.031841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:50.096573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:16:50.098809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0714ad7-e792-435f-8789-8707e113f182 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:16:50.121102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 648/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:16:52.419812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:52.420349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:52.420458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:52.422117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 664/1312] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:16:52.428640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44991-52e1-4167-baf0-16c4f0cff684 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:52.429193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44991-52e1-4167-baf0-16c4f0cff684 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:52.429193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44991-52e1-4167-baf0-16c4f0cff684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:52.509411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-28a44991-52e1-4167-baf0-16c4f0cff684 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:16:52.510609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-28a44991-52e1-4167-baf0-16c4f0cff684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:52.615791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 649/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:16:52.653225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:16:52.654877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:52.765976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5729536-61ad-479c-b1e3-af7bd0359bdc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:16:52.848958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:52.852116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:16:52.852923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:52.853258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:52.853906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:52.856112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:52.966815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:16:52.968304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3be43b-8d9f-4803-a1d6-fb564597c7e7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:16:52.986466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 665/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:16:55.325101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:55.325600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:55.325720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:55.327022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 650/1315] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:16:55.332048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640986f2-f34c-41ab-9870-02d924d22b1e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:55.332452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640986f2-f34c-41ab-9870-02d924d22b1e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:55.332648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640986f2-f34c-41ab-9870-02d924d22b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:55.395707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-640986f2-f34c-41ab-9870-02d924d22b1e None None] MFA Rules not processed for user `a61f6086b6a94120b20b884d3e0d010e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:16:55.396922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-640986f2-f34c-41ab-9870-02d924d22b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:55.449094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:55.493881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 666/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:16:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:16:55.532956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:16:55.534494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:55.544928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:55.636406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060eb8-847a-495c-82da-f16893ac21a6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:16:55.708438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:16:55.709078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:55.709380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:55.709641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:55.711277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:55.770815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:55.817588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:16:55.818947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11eb2d7-ccc5-499c-b284-34e49462bfcc service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:16:55.836199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 651/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:16:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:16:58.370030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:58.370584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:58.370584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:58.372112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 667/1318] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:16:58.377267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c99-c740-475e-8564-94ebef5281e7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:58.377666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c99-c740-475e-8564-94ebef5281e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:58.377838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c9c99-c740-475e-8564-94ebef5281e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:58.442152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-847c9c99-c740-475e-8564-94ebef5281e7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:16:58.443423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-847c9c99-c740-475e-8564-94ebef5281e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:58.538160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 652/1319] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:16:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:16:58.575784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:16:58.577336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:58.679609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3d3bf-4fc0-4f1f-bd77-72d6f3165f54 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:16:58.749934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:16:58.750559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:16:58.750841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:16:58.751115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:16:58.752763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:16:58.760428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:16:58.860199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:16:58.861640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46033928-5777-4d5d-a6b9-30e52962bbd1 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:16:58.880640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 668/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:16:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:01.533956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f877-94d1-4f0d-8b2e-269243a19c12 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:01.534470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f877-94d1-4f0d-8b2e-269243a19c12 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:01.534470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207f877-94d1-4f0d-8b2e-269243a19c12 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:01.535875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 653/1321] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:01.541264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b35b-4c55-47b0-bb3d-09c530cabaa7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:01.541714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b35b-4c55-47b0-bb3d-09c530cabaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:01.541836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b35b-4c55-47b0-bb3d-09c530cabaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:01.605618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b345b35b-4c55-47b0-bb3d-09c530cabaa7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:01.606823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b345b35b-4c55-47b0-bb3d-09c530cabaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:01.668887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:01.721168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 669/1322] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:17:01.773875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:01.774169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:01.774457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:01.775940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 654/1323] 10.222.0.46 () {54 vars in 964 bytes} [Wed Jul 6 15:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:01.782488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67961d-d1c7-4e4a-82f9-8ad31a4e440c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:01.782783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67961d-d1c7-4e4a-82f9-8ad31a4e440c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:01.783064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67961d-d1c7-4e4a-82f9-8ad31a4e440c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:01.864471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6f67961d-d1c7-4e4a-82f9-8ad31a4e440c None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:01.865679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6f67961d-d1c7-4e4a-82f9-8ad31a4e440c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:01.943364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 670/1324] 10.222.0.46 () {58 vars in 1061 bytes} [Wed Jul 6 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:01.952203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:01.953735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:01.973156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:02.056802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa2b5-5377-483a-b92d-84a9219ed2e6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:02.128664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:02.129242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:02.129529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:02.129805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:02.131486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:02.202143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:02.241625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:02.244238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f838a5-7be0-4f4b-b4fd-ac216e504af5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:02.265590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 655/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:04.574422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74377641-d586-46bd-906b-d22382265c4a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:04.574967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74377641-d586-46bd-906b-d22382265c4a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:04.574967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74377641-d586-46bd-906b-d22382265c4a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:04.576349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 671/1326] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:04.581500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f16-95ef-4a38-b5a1-1f79ada8baa8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:04.581940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f16-95ef-4a38-b5a1-1f79ada8baa8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:04.582048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea923f16-95ef-4a38-b5a1-1f79ada8baa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:04.644732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ea923f16-95ef-4a38-b5a1-1f79ada8baa8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:04.645941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ea923f16-95ef-4a38-b5a1-1f79ada8baa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:04.751141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 656/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:04.786825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74377641-d586-46bd-906b-d22382265c4a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:04.788754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-74377641-d586-46bd-906b-d22382265c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:04.849840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:04.895567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74377641-d586-46bd-906b-d22382265c4a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:04.970201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:04.970856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:04.971142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:04.971450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:04.973620 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:05.081894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:05.084442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:05.086427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220c3c9-8fc7-40b7-b2af-89f967200a4d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:05.105571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 672/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:07.179462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:07.179971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:07.180085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:07.181389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 657/1329] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:07.186707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf66f-6216-4a1f-8472-f33c8e6c2869 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:07.187110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf66f-6216-4a1f-8472-f33c8e6c2869 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:07.187319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf66f-6216-4a1f-8472-f33c8e6c2869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:07.252127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-06fbf66f-6216-4a1f-8472-f33c8e6c2869 None None] MFA Rules not processed for user `a61f6086b6a94120b20b884d3e0d010e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:07.253370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06fbf66f-6216-4a1f-8472-f33c8e6c2869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:07.367941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 673/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:07.405492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:07.407023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:07.437227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:07.511173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c951e-8336-4824-9230-8fa12fdbc607 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:07.583062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:07.583740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:07.584017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:07.584293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:07.585926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:07.664797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:07.698050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:07.699531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa0331-016d-47f6-b845-314a37b89c4c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:07.717796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 658/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:10.163930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:10.164465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:10.164579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:10.165888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 674/1332] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:10.170928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e981-57b3-49aa-a805-ae080d99baf7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:10.171449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e981-57b3-49aa-a805-ae080d99baf7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:10.171546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e981-57b3-49aa-a805-ae080d99baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:10.234704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-55f8e981-57b3-49aa-a805-ae080d99baf7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:10.235943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-55f8e981-57b3-49aa-a805-ae080d99baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:10.336370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 659/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:10.370881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:10.372468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:10.441664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:10.475767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775542c-f337-4d65-9ce6-889d7d42cf42 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:10.547606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:10.548296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:10.548585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:10.548865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:10.550990 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:10.662000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:10.663731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc06d7-50c1-4e7f-affa-d7069ebc558d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:10.673558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:10.682580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 675/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:12.740994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:12.741572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:12.741572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:12.742903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 660/1335] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:17:12.748151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575a56-954e-413d-8b0f-80a942f03618 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:12.748612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575a56-954e-413d-8b0f-80a942f03618 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:12.748714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575a56-954e-413d-8b0f-80a942f03618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:12.812107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6c575a56-954e-413d-8b0f-80a942f03618 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:12.813323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c575a56-954e-413d-8b0f-80a942f03618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:12.908648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 676/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:12.947893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:12.949407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:12.989306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:13.052118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aac163-a75d-46ae-bb7e-cde27ffab754 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:13.122441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:13.123082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:13.123398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:13.123675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:13.125300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:13.205865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:13.231495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:13.232933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13c81e-3813-4d28-b929-755df46f260a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:13.252219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 661/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:15.558112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:15.558632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:15.558632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:15.560026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 677/1338] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:15.565417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324e5a-1cdf-45c5-a75c-e668c5d4ac80 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:15.566353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324e5a-1cdf-45c5-a75c-e668c5d4ac80 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:15.566484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324e5a-1cdf-45c5-a75c-e668c5d4ac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:15.629821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-72324e5a-1cdf-45c5-a75c-e668c5d4ac80 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:15.631037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-72324e5a-1cdf-45c5-a75c-e668c5d4ac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:15.726173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 662/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:15.761795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:15.763368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:15.838320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:15.865854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575de603-2a72-4562-bd60-e188f9ca3fbb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:15.938089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:15.938739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:15.939024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:15.939331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:15.941567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:16.052952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:16.054388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d23af-c6ea-41b6-a90b-1864f9c363d9 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:16.073054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 678/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:20.020728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:20.021248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:20.021248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:20.022597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 663/1341] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:20.027847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5fe8-269a-42c3-9371-91842b883fe8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:20.028268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5fe8-269a-42c3-9371-91842b883fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:20.028436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f5fe8-269a-42c3-9371-91842b883fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:20.034608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:20.099234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-061f5fe8-269a-42c3-9371-91842b883fe8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:20.100498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-061f5fe8-269a-42c3-9371-91842b883fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:20.203268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 679/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:20.241226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:20.243652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:20.298651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:20.347250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29526e75-630d-4e12-88a3-5c6f91833b09 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:20.350889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:20.351692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:20.351981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:20.352260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:20.361377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:20.363420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8891c6-6eef-472d-8fbd-661d664cf0f8 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:20.372755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 664/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:23.052101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:23.052626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:23.052818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:23.054093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 680/1344] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:23.059387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e43f-396e-426e-9ec0-02b28c44a958 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:23.059772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e43f-396e-426e-9ec0-02b28c44a958 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:23.060061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304e43f-396e-426e-9ec0-02b28c44a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:23.087046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4304e43f-396e-426e-9ec0-02b28c44a958 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:23.088190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4304e43f-396e-426e-9ec0-02b28c44a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:23.171405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 665/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:23.209595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:23.211445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:23.225483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649f8f1-4edc-4353-9bf0-84189c5d188f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:23.229207 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:23.229880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:23.230170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:23.230453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:23.241116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:23.242729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a97dbc-f4be-414e-8762-4fab19aec448 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:23.256634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 681/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:27.604689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a6003-1045-4a25-bf7f-d820781c1625 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:27.605197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a6003-1045-4a25-bf7f-d820781c1625 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:27.605197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a6003-1045-4a25-bf7f-d820781c1625 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:27.606549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 666/1347] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:27.611736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45aa7-133b-47cf-821c-c16b290fc677 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:27.612186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45aa7-133b-47cf-821c-c16b290fc677 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:27.612501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45aa7-133b-47cf-821c-c16b290fc677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:27.632570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-01e45aa7-133b-47cf-821c-c16b290fc677 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:27.633789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-01e45aa7-133b-47cf-821c-c16b290fc677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:27.641877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 682/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:27.679029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a6003-1045-4a25-bf7f-d820781c1625 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:27.689712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a6003-1045-4a25-bf7f-d820781c1625 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:27.693788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:27.694557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:27.694911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:27.695376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:27.705632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:27.707683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08851627-cc1a-432a-a69c-2f1867944581 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:27.718048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 667/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:32.690377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b212eaf-dd79-4164-9ae6-95ec71eb5382 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:32.690881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b212eaf-dd79-4164-9ae6-95ec71eb5382 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:32.691509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b212eaf-dd79-4164-9ae6-95ec71eb5382 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:32.694636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 683/1350] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:17:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:32.699853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4aed-3d33-4e86-8ed7-592a215e69e8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:32.700300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4aed-3d33-4e86-8ed7-592a215e69e8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:32.700409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e4aed-3d33-4e86-8ed7-592a215e69e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:32.784962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6b1e4aed-3d33-4e86-8ed7-592a215e69e8 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:32.786071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6b1e4aed-3d33-4e86-8ed7-592a215e69e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:32.814678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:32.866167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 668/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:17:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:36.899709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7cdb-04dd-4aa7-aa64-4835f267c820 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:36.901411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7cdb-04dd-4aa7-aa64-4835f267c820 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:36.901710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7cdb-04dd-4aa7-aa64-4835f267c820 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:36.903170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 684/1352] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:17:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:36.909098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4876-7ca1-425d-b842-48ef188569d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:36.909905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4876-7ca1-425d-b842-48ef188569d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:36.910263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4876-7ca1-425d-b842-48ef188569d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:36.930028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e8cf4876-7ca1-425d-b842-48ef188569d6 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:36.931664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8cf4876-7ca1-425d-b842-48ef188569d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:36.941055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 669/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:17:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:37.625146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=72 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:38.467439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710510-5ae5-4af0-ab60-560fee8fc772 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:38.468110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710510-5ae5-4af0-ab60-560fee8fc772 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:38.468110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710510-5ae5-4af0-ab60-560fee8fc772 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:38.469523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 685/1354] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:38.475417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754d883-a86f-4a54-8ade-268f94ee5fe7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:38.475834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754d883-a86f-4a54-8ade-268f94ee5fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:38.475977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754d883-a86f-4a54-8ade-268f94ee5fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:38.493607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a754d883-a86f-4a54-8ade-268f94ee5fe7 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:38.494816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a754d883-a86f-4a54-8ade-268f94ee5fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:38.503648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 670/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:17:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:17:42.148646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f9f6-6e71-43d8-9fbd-0b95badb4b87 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:42.149543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f9f6-6e71-43d8-9fbd-0b95badb4b87 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:42.149978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f9f6-6e71-43d8-9fbd-0b95badb4b87 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:42.151866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 686/1356] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:42.157327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f3c9-cd78-4f36-91ef-21a38c1c011f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:42.157807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f3c9-cd78-4f36-91ef-21a38c1c011f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:42.157914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4f3c9-cd78-4f36-91ef-21a38c1c011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:42.193410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c8b4f3c9-cd78-4f36-91ef-21a38c1c011f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:42.196065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c8b4f3c9-cd78-4f36-91ef-21a38c1c011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:42.211255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 671/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:42.271071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbae11-8057-4f34-9516-ec0cb6fc5705 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:42.271760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbae11-8057-4f34-9516-ec0cb6fc5705 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:42.272320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbae11-8057-4f34-9516-ec0cb6fc5705 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:42.273730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 687/1358] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 6 15:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:42.286689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609829-7cef-46ad-8004-2442945d530e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:42.287795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609829-7cef-46ad-8004-2442945d530e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:42.288739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab609829-7cef-46ad-8004-2442945d530e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:42.343335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ab609829-7cef-46ad-8004-2442945d530e None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:42.344781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab609829-7cef-46ad-8004-2442945d530e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:42.430364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 672/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 6 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:42.438671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbae11-8057-4f34-9516-ec0cb6fc5705 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:42.453921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbae11-8057-4f34-9516-ec0cb6fc5705 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:42.460054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:42.460801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:42.461182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:42.461554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:42.471996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:42.473645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ae72b-125d-4981-998f-631b56baacd3 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:42.483916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 688/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:17:44.714622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238b82-e43a-4783-9ad4-a68ed82a5265 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:44.715129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238b82-e43a-4783-9ad4-a68ed82a5265 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:44.715344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238b82-e43a-4783-9ad4-a68ed82a5265 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:44.716597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 673/1361] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:44.722089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b307a-dc36-4467-9b9d-cd03bad92092 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:44.722596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b307a-dc36-4467-9b9d-cd03bad92092 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:44.722982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b307a-dc36-4467-9b9d-cd03bad92092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:44.740836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ed0b307a-dc36-4467-9b9d-cd03bad92092 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:44.742159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed0b307a-dc36-4467-9b9d-cd03bad92092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:44.750311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 689/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:44.795106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238b82-e43a-4783-9ad4-a68ed82a5265 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:44.809820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238b82-e43a-4783-9ad4-a68ed82a5265 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:44.816104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:44.816856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:44.817218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:44.817579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:44.826199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:44.827929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eca18b-734e-4aa9-aad9-211e4504b252 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:44.845811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 674/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:47.190379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fda3-cfdb-4278-bd2e-59e045934316 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:47.190902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fda3-cfdb-4278-bd2e-59e045934316 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:47.190902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9fda3-cfdb-4278-bd2e-59e045934316 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:47.192628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 690/1364] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:47.197871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0884ebb-21be-44bd-aeea-398b1dde8299 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:47.198284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0884ebb-21be-44bd-aeea-398b1dde8299 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:47.198579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0884ebb-21be-44bd-aeea-398b1dde8299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:47.219603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a0884ebb-21be-44bd-aeea-398b1dde8299 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:47.221049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a0884ebb-21be-44bd-aeea-398b1dde8299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:47.228915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 675/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:47.279409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585d5f-b3e4-443a-afa4-dfc2d4ea3b62 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:47.279409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585d5f-b3e4-443a-afa4-dfc2d4ea3b62 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:47.279409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585d5f-b3e4-443a-afa4-dfc2d4ea3b62 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:47.281295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 691/1366] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 6 15:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:47.287422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453426-1b7c-44cb-85aa-a22a72363c40 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:47.287725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453426-1b7c-44cb-85aa-a22a72363c40 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:47.288013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7453426-1b7c-44cb-85aa-a22a72363c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:47.308181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f7453426-1b7c-44cb-85aa-a22a72363c40 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:47.309591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f7453426-1b7c-44cb-85aa-a22a72363c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:47.326309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 676/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 6 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:47.334261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585d5f-b3e4-443a-afa4-dfc2d4ea3b62 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:47.350204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585d5f-b3e4-443a-afa4-dfc2d4ea3b62 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:47.356437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:47.357045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:47.357327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:47.357600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:47.366914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:47.368530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e4520-756e-4524-b241-0c90dbeb5e50 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:47.378579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 692/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:49.450638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f44fd-3b60-4a5f-b839-3d903d666fee None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:49.451174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f44fd-3b60-4a5f-b839-3d903d666fee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:49.451311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f44fd-3b60-4a5f-b839-3d903d666fee None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:49.452619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 677/1369] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:49.457979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53698b-4d88-40fc-9123-ca9a8224f387 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:49.458403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53698b-4d88-40fc-9123-ca9a8224f387 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:49.458740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53698b-4d88-40fc-9123-ca9a8224f387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:49.477613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-df53698b-4d88-40fc-9123-ca9a8224f387 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:49.478825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-df53698b-4d88-40fc-9123-ca9a8224f387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:49.487757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 693/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:49.538303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f44fd-3b60-4a5f-b839-3d903d666fee None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:49.552872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f44fd-3b60-4a5f-b839-3d903d666fee None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:49.558397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:49.559045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:49.559366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:49.559652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:49.568289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:49.570184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeab3ad8-438d-4a72-b8ba-ff000e87823e service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:49.580385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 678/1371] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:50.313575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2288f-6755-49d6-bdf5-2164be5ca5b1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:50.314747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2288f-6755-49d6-bdf5-2164be5ca5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:50.315053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2288f-6755-49d6-bdf5-2164be5ca5b1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:50.316760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 694/1372] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:50.321974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5548-7f9e-4923-bfac-9d1ae4fc8741 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:50.322268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5548-7f9e-4923-bfac-9d1ae4fc8741 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:50.322548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5548-7f9e-4923-bfac-9d1ae4fc8741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:50.340622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4dfe5548-7f9e-4923-bfac-9d1ae4fc8741 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:50.341811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4dfe5548-7f9e-4923-bfac-9d1ae4fc8741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:50.351385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 679/1373] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:50.684245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835440c1-6f3d-45f6-aa6f-9685fdc799ba None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:50.684870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835440c1-6f3d-45f6-aa6f-9685fdc799ba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:50.684870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835440c1-6f3d-45f6-aa6f-9685fdc799ba None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:50.687256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 695/1374] 10.222.0.46 () {54 vars in 963 bytes} [Wed Jul 6 15:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:50.691117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a059-da02-417c-aef9-e2eb3329619f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:50.691684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a059-da02-417c-aef9-e2eb3329619f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:50.691684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58a059-da02-417c-aef9-e2eb3329619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:50.698120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:17:50.744365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ce58a059-da02-417c-aef9-e2eb3329619f None None] MFA Rules not processed for user `edbb85bda10d45bf8f66c6ee0561c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:50.745589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce58a059-da02-417c-aef9-e2eb3329619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:50.806715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 680/1375] 10.222.0.46 () {58 vars in 1060 bytes} [Wed Jul 6 15:17:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:50.814692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835440c1-6f3d-45f6-aa6f-9685fdc799ba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:50.825573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835440c1-6f3d-45f6-aa6f-9685fdc799ba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:50.829087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:50.829800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:50.830154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:50.830498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:50.839800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:50.841454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca00d03-9d2e-43a9-979a-e60289d2fa85 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:50.852657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 696/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:51.656694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fb0e-7176-4619-90da-5cfd7e8f249e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:51.657061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fb0e-7176-4619-90da-5cfd7e8f249e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:51.657409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35fb0e-7176-4619-90da-5cfd7e8f249e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:51.659543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 681/1377] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 15:17:51.667598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf96ad3-4bc5-4624-9fa2-3b4b9d790e74 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:51.669878 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf96ad3-4bc5-4624-9fa2-3b4b9d790e74 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:51.670183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf96ad3-4bc5-4624-9fa2-3b4b9d790e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:51.689520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-edf96ad3-4bc5-4624-9fa2-3b4b9d790e74 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:51.690772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-edf96ad3-4bc5-4624-9fa2-3b4b9d790e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:51.699886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 697/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:51.744094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35fb0e-7176-4619-90da-5cfd7e8f249e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:51.760093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35fb0e-7176-4619-90da-5cfd7e8f249e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:51.765900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:51.766695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:51.767576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:51.767932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:51.779179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:51.780890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c297409-ad45-4243-882c-a4754d77fa8c service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:51.792142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 682/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:53.849234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a675b-58ac-43f7-8cb9-c957755b9f9a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:53.849761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a675b-58ac-43f7-8cb9-c957755b9f9a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:53.849899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a675b-58ac-43f7-8cb9-c957755b9f9a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:53.851277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 698/1380] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:53.856412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d043f-eb65-4cc2-a187-057e881deafe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:53.857056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d043f-eb65-4cc2-a187-057e881deafe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:53.857056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d043f-eb65-4cc2-a187-057e881deafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:53.899076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e61d043f-eb65-4cc2-a187-057e881deafe None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:53.904873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e61d043f-eb65-4cc2-a187-057e881deafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:53.913299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 683/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:53.958584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a675b-58ac-43f7-8cb9-c957755b9f9a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:53.974320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a675b-58ac-43f7-8cb9-c957755b9f9a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:53.980037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:53.981979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:53.982269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:53.982555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:53.990817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:53.992512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e2759-f92d-49ef-bc32-5ca1fc9e7fe2 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:54.005398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 699/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:56.071197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc5f4-86d7-4cbe-89a8-ff3ac58803d6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:56.071693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc5f4-86d7-4cbe-89a8-ff3ac58803d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:56.072023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc5f4-86d7-4cbe-89a8-ff3ac58803d6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:56.073872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 684/1383] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:56.078729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450d36a-6a60-4d01-8933-ac4b7fd1cee6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:56.079186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450d36a-6a60-4d01-8933-ac4b7fd1cee6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:56.079373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450d36a-6a60-4d01-8933-ac4b7fd1cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:56.097746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a450d36a-6a60-4d01-8933-ac4b7fd1cee6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:56.098943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a450d36a-6a60-4d01-8933-ac4b7fd1cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:56.106985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 700/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:17:56.148281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fc5f4-86d7-4cbe-89a8-ff3ac58803d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:56.163694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fc5f4-86d7-4cbe-89a8-ff3ac58803d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:56.169159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:56.169898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:56.170357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:56.170718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:56.180429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:56.182122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30355904-28c4-485a-a4e1-338119376d1b service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:56.192371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 685/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:17:58.230377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85a12-e590-40f4-8961-6b650b8bdd03 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:58.231152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85a12-e590-40f4-8961-6b650b8bdd03 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:58.231152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85a12-e590-40f4-8961-6b650b8bdd03 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:58.237516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb15c-0d0e-41b3-a293-3df8c2605cac None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:58.237516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb15c-0d0e-41b3-a293-3df8c2605cac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:58.245534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbb15c-0d0e-41b3-a293-3df8c2605cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:58.245812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 701/1386] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:17:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:17:58.257533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b9cbb15c-0d0e-41b3-a293-3df8c2605cac None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:17:58.258712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b9cbb15c-0d0e-41b3-a293-3df8c2605cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:17:58.267678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 686/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:17:58.312661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad85a12-e590-40f4-8961-6b650b8bdd03 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:17:58.327513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad85a12-e590-40f4-8961-6b650b8bdd03 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:17:58.335454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:17:58.336054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:17:58.336342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:17:58.336622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:17:58.347326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:17:58.349215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4c3e7-31c7-414e-bb8e-3b6d4ceb04b1 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:17:58.359453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 702/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:00.383450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d544-6ba6-4ca6-bccd-21af2886ccb3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:00.383997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d544-6ba6-4ca6-bccd-21af2886ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:00.384081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d544-6ba6-4ca6-bccd-21af2886ccb3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:00.385514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 687/1389] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:00.390500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c0f-35a5-4783-ae74-e54c658e5c53 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:00.390990 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c0f-35a5-4783-ae74-e54c658e5c53 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:00.391176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f1c0f-35a5-4783-ae74-e54c658e5c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:00.409384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-016f1c0f-35a5-4783-ae74-e54c658e5c53 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:00.410571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-016f1c0f-35a5-4783-ae74-e54c658e5c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:00.418596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 703/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:00.462981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3d544-6ba6-4ca6-bccd-21af2886ccb3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:00.482101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3d544-6ba6-4ca6-bccd-21af2886ccb3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:00.489761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:00.490686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:00.491053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:00.491446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:00.501978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:00.504230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b92d-70d9-4903-a417-e75c51c9eb53 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:00.516610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 688/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:18:02.558266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1554f-97e9-4203-b7cf-54c39bd7bbe7 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:02.558791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1554f-97e9-4203-b7cf-54c39bd7bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:02.558791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1554f-97e9-4203-b7cf-54c39bd7bbe7 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:02.560314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 704/1392] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:02.567383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6ddb-6ad8-4d03-bfb8-92704fa034bd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:02.567814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6ddb-6ad8-4d03-bfb8-92704fa034bd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:02.568093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6ddb-6ad8-4d03-bfb8-92704fa034bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:02.591207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-52cf6ddb-6ad8-4d03-bfb8-92704fa034bd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:02.592867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-52cf6ddb-6ad8-4d03-bfb8-92704fa034bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:02.602924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 689/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:02.647861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1554f-97e9-4203-b7cf-54c39bd7bbe7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:02.663571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1554f-97e9-4203-b7cf-54c39bd7bbe7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:02.669220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:02.669893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:02.670196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:02.670488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:02.679705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:02.681334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035f15b-fb6b-483a-ada8-ebe48f71a803 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:02.691634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 705/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:04.718968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cc11-a8f0-4db3-af6c-7c4a7783d562 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:04.719563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cc11-a8f0-4db3-af6c-7c4a7783d562 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:04.719563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7cc11-a8f0-4db3-af6c-7c4a7783d562 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:04.720942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 690/1395] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:04.726164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd96df-cff0-4254-bb27-f1a319e945c0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:04.726561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd96df-cff0-4254-bb27-f1a319e945c0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:04.726722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd96df-cff0-4254-bb27-f1a319e945c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:04.744525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e8bd96df-cff0-4254-bb27-f1a319e945c0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:04.745706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8bd96df-cff0-4254-bb27-f1a319e945c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:04.754009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 706/1396] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:04.796158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7cc11-a8f0-4db3-af6c-7c4a7783d562 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:04.810792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7cc11-a8f0-4db3-af6c-7c4a7783d562 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:04.816606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:04.817243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:04.817525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:04.817801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:04.825786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:04.827384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a986a6-364a-4721-b748-bef0e1c0642d service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:04.836818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 691/1397] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:05.959856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846405-9a71-41d6-ae48-76399f528a2d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:05.960445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846405-9a71-41d6-ae48-76399f528a2d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:05.960445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8846405-9a71-41d6-ae48-76399f528a2d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:05.961826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 707/1398] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:05.967643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1ba5-cbaf-4c0c-b9d7-980a7c8169ce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:05.968133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1ba5-cbaf-4c0c-b9d7-980a7c8169ce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:05.968249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc1ba5-cbaf-4c0c-b9d7-980a7c8169ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:05.985995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f9cc1ba5-cbaf-4c0c-b9d7-980a7c8169ce None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:05.987197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f9cc1ba5-cbaf-4c0c-b9d7-980a7c8169ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:05.995685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 692/1399] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:06.051748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdc6b-b76d-409e-9b1b-24089c17f39a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.052048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdc6b-b76d-409e-9b1b-24089c17f39a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.052331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bdc6b-b76d-409e-9b1b-24089c17f39a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.053705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 708/1400] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 6 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:06.058699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887864a1-eab7-4beb-899a-2dbce558f2e4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.058994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887864a1-eab7-4beb-899a-2dbce558f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.059301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887864a1-eab7-4beb-899a-2dbce558f2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.105929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-887864a1-eab7-4beb-899a-2dbce558f2e4 None None] MFA Rules not processed for user `f11ea126550c4c4a8892522e01ed149a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:06.107108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-887864a1-eab7-4beb-899a-2dbce558f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:06.139402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:18:06.167203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 693/1401] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 6 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:06.174995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bdc6b-b76d-409e-9b1b-24089c17f39a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:06.185616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bdc6b-b76d-409e-9b1b-24089c17f39a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:06.189791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:06.190434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.190727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.191012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.200059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:06.201487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac37bd-f4cf-456f-b7ba-96834807da2e service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:06.211080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 709/1402] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:06.885299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29260-cd3a-48e7-91d4-1a16dc96a113 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.885655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29260-cd3a-48e7-91d4-1a16dc96a113 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.885940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29260-cd3a-48e7-91d4-1a16dc96a113 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.887338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 694/1403] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:06.893125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.893431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.893714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.912387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:06.913578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:06.917323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876857-eb01-408b-bd27-489887f17ab1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.917619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876857-eb01-408b-bd27-489887f17ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.917904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29876857-eb01-408b-bd27-489887f17ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.921893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 710/1404] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:06.966091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-29876857-eb01-408b-bd27-489887f17ab1 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:06.967345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-29876857-eb01-408b-bd27-489887f17ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:06.967619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:06.980801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59e5cf-b373-435a-8164-c9b989b620e0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:06.986380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:06.987034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:06.987356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:06.987642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:06.997886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:06.999546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942aa81-53ba-4299-a53e-33efc8d418a4 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:07.009114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 711/1405] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:07.036042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 695/1406] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:18:07.075546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ad8b-dc0b-48d2-af1e-44bd9669e9fd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:07.075947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ad8b-dc0b-48d2-af1e-44bd9669e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:07.076298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137ad8b-dc0b-48d2-af1e-44bd9669e9fd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:07.077815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 712/1407] 10.222.0.46 () {54 vars in 942 bytes} [Wed Jul 6 15:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:07.086246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c967b4-7931-4c48-845a-f162ade98efb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:07.086545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c967b4-7931-4c48-845a-f162ade98efb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:07.086830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c967b4-7931-4c48-845a-f162ade98efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:07.141061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-71c967b4-7931-4c48-845a-f162ade98efb None None] MFA Rules not processed for user `44f05b05c1c94c44aa499f224ac20648`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:07.142846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-71c967b4-7931-4c48-845a-f162ade98efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:07.219365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 696/1408] 10.222.0.46 () {58 vars in 1039 bytes} [Wed Jul 6 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:07.228780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ad8b-dc0b-48d2-af1e-44bd9669e9fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:07.245038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137ad8b-dc0b-48d2-af1e-44bd9669e9fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:07.252958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:07.253172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:07.253462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:07.253736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:07.262700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:07.264568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e0fd-2fec-4447-b01b-056fb1f12918 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:07.273610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 713/1409] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:18:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:18:17.216146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:18:17.270263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=75 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:18:29.220725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e465d99-8296-4213-ab0d-7db2720c37e5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:29.221289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e465d99-8296-4213-ab0d-7db2720c37e5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:29.221289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e465d99-8296-4213-ab0d-7db2720c37e5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:29.222670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 697/1410] 10.222.0.46 () {54 vars in 918 bytes} [Wed Jul 6 15:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:29.227815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf3a-7ff3-4124-b663-feebf10323fa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:29.228220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf3a-7ff3-4124-b663-feebf10323fa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:29.228382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf3a-7ff3-4124-b663-feebf10323fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:29.246967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-451ecf3a-7ff3-4124-b663-feebf10323fa None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:29.248235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-451ecf3a-7ff3-4124-b663-feebf10323fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:29.257440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 714/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jul 6 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:29.271595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8bd42-fbad-46d8-a8c3-3fae5192c98e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:29.271891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8bd42-fbad-46d8-a8c3-3fae5192c98e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:29.272199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8bd42-fbad-46d8-a8c3-3fae5192c98e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:29.273558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 698/1412] 10.222.0.46 () {54 vars in 918 bytes} [Wed Jul 6 15:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:29.278112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa32c4-ad18-418c-a24d-f4b2727f0a3c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:29.278404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa32c4-ad18-418c-a24d-f4b2727f0a3c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:29.278689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa32c4-ad18-418c-a24d-f4b2727f0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:29.296559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-66aa32c4-ad18-418c-a24d-f4b2727f0a3c None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:29.298225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66aa32c4-ad18-418c-a24d-f4b2727f0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:29.307297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 715/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jul 6 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:18:31.751380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855010-2a3e-4488-bac7-d88cee055d11 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:31.751943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855010-2a3e-4488-bac7-d88cee055d11 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:31.751943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76855010-2a3e-4488-bac7-d88cee055d11 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:31.753294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:31.758405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce23d7-420e-453d-a783-9f8e80e0a34d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:31.758810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce23d7-420e-453d-a783-9f8e80e0a34d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:31.759001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce23d7-420e-453d-a783-9f8e80e0a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:31.776760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-50ce23d7-420e-453d-a783-9f8e80e0a34d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:31.777953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50ce23d7-420e-453d-a783-9f8e80e0a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:31.786484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:31.830549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76855010-2a3e-4488-bac7-d88cee055d11 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:31.845873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76855010-2a3e-4488-bac7-d88cee055d11 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:31.851171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:31.851952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:31.852241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:31.852519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:31.860675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:31.862301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41561e3d-0199-4f15-9a5a-50eacce84aad service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:31.871548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 700/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:32.755044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb783e4-8159-45dd-8fe2-2867c427a519 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:32.755378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb783e4-8159-45dd-8fe2-2867c427a519 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:32.755665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb783e4-8159-45dd-8fe2-2867c427a519 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:32.757060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:32.761942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d759-871c-4662-8ea3-b274cdfe086d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:32.762405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d759-871c-4662-8ea3-b274cdfe086d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:32.762528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930d759-871c-4662-8ea3-b274cdfe086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:32.780430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3930d759-871c-4662-8ea3-b274cdfe086d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:32.781649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3930d759-871c-4662-8ea3-b274cdfe086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:32.790275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:32.939309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb783e4-8159-45dd-8fe2-2867c427a519 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:32.953419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb783e4-8159-45dd-8fe2-2867c427a519 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:32.958729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:32.959405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:32.959695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:32.959978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:32.967948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:32.970038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ed8e7-5e48-495b-8ab8-233c64ee41eb service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:32.979189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 718/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:33.851401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a33f-cb6c-401a-bef0-9901f281b139 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:33.851964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a33f-cb6c-401a-bef0-9901f281b139 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:33.851964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94a33f-cb6c-401a-bef0-9901f281b139 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:33.853342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:33.858113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492b723-8a7d-4b5a-a0cd-660ace05416d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:33.858413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492b723-8a7d-4b5a-a0cd-660ace05416d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:33.858699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492b723-8a7d-4b5a-a0cd-660ace05416d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:33.878257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6492b723-8a7d-4b5a-a0cd-660ace05416d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:33.879499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6492b723-8a7d-4b5a-a0cd-660ace05416d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:33.887424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:34.038618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94a33f-cb6c-401a-bef0-9901f281b139 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:34.053538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94a33f-cb6c-401a-bef0-9901f281b139 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:34.058784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:34.059450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:34.059736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:34.060009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:34.068491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:34.070065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013da67b-7ee6-4785-a2ef-45c6e151fe15 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:34.079320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 703/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:34.931811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85763e7-c460-4b67-abc6-7bb41645d4a8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:34.932342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85763e7-c460-4b67-abc6-7bb41645d4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:34.932458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85763e7-c460-4b67-abc6-7bb41645d4a8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:34.933769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:34.939234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93ab10-09e9-4d39-83d6-9a65c02c1d58 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:34.939631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93ab10-09e9-4d39-83d6-9a65c02c1d58 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:34.939837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93ab10-09e9-4d39-83d6-9a65c02c1d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:34.958062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2b93ab10-09e9-4d39-83d6-9a65c02c1d58 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:34.959338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b93ab10-09e9-4d39-83d6-9a65c02c1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:34.967116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:35.116137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85763e7-c460-4b67-abc6-7bb41645d4a8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:35.130649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85763e7-c460-4b67-abc6-7bb41645d4a8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:35.135991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:35.136631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:35.136924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:35.137203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:35.145898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:35.147504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e98b43-747a-4aaf-b197-4d9509197092 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:35.156730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 721/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:36.027770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb785d1e-6ce0-4b0b-88b7-cba0676850d5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:36.028349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb785d1e-6ce0-4b0b-88b7-cba0676850d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:36.028349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb785d1e-6ce0-4b0b-88b7-cba0676850d5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:36.029708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:36.035376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7def91-e9b4-4b07-995b-01a721b89cb8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:36.035817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7def91-e9b4-4b07-995b-01a721b89cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:36.035924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7def91-e9b4-4b07-995b-01a721b89cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:36.054167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ac7def91-e9b4-4b07-995b-01a721b89cb8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:36.055395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ac7def91-e9b4-4b07-995b-01a721b89cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:36.063064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:36.209093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb785d1e-6ce0-4b0b-88b7-cba0676850d5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:36.223694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb785d1e-6ce0-4b0b-88b7-cba0676850d5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:36.228930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:36.229566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:36.229851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:36.230128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:36.238139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:36.239754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc82c6f-23e4-412a-9ac8-9a04644d4c56 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:36.249295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 706/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:37.039889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc0c2-5bb0-4814-8fc5-e4b273ac1c6f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:37.040441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc0c2-5bb0-4814-8fc5-e4b273ac1c6f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:37.040441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc0c2-5bb0-4814-8fc5-e4b273ac1c6f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:37.041846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:37.046584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed376c-af63-4911-aad3-5b176f178891 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:37.047025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed376c-af63-4911-aad3-5b176f178891 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:37.047654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed376c-af63-4911-aad3-5b176f178891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:37.065814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-22ed376c-af63-4911-aad3-5b176f178891 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:37.067011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22ed376c-af63-4911-aad3-5b176f178891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:37.074723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:37.223241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc0c2-5bb0-4814-8fc5-e4b273ac1c6f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:37.238030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc0c2-5bb0-4814-8fc5-e4b273ac1c6f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:37.243424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:37.244076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:37.244358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:37.244634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:37.253257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:37.254839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56b00b-34aa-4396-8718-489363e97120 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:37.264142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 724/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:38.060088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a384-716a-4cea-b782-ba378bf55510 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:38.060647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a384-716a-4cea-b782-ba378bf55510 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:38.060647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a384-716a-4cea-b782-ba378bf55510 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:38.062025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:38.067393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c5f30-4ba5-4a8c-8200-ebec98cd83ac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:38.067814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c5f30-4ba5-4a8c-8200-ebec98cd83ac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:38.067955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c5f30-4ba5-4a8c-8200-ebec98cd83ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:38.086163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-263c5f30-4ba5-4a8c-8200-ebec98cd83ac None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:38.087374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-263c5f30-4ba5-4a8c-8200-ebec98cd83ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:38.094913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:38.243675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2a384-716a-4cea-b782-ba378bf55510 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:38.265717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2a384-716a-4cea-b782-ba378bf55510 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:38.269258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:38.269916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:38.270205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:38.270484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:38.278839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:38.280457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03122e16-b765-4d07-a165-5a3e7a94740f service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:38.290661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 709/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:39.085135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d59a33-53bc-4edb-8c60-0067a23de599 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:39.085698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d59a33-53bc-4edb-8c60-0067a23de599 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:39.085698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d59a33-53bc-4edb-8c60-0067a23de599 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:39.087069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:39.091838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f27ac-6573-4fc2-ba12-037d46feaac0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:39.092266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f27ac-6573-4fc2-ba12-037d46feaac0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:39.092399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f27ac-6573-4fc2-ba12-037d46feaac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:39.110486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-366f27ac-6573-4fc2-ba12-037d46feaac0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:39.111784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-366f27ac-6573-4fc2-ba12-037d46feaac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:39.119498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:39.267133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d59a33-53bc-4edb-8c60-0067a23de599 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:39.281049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d59a33-53bc-4edb-8c60-0067a23de599 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:39.286572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:39.287206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:39.287513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:39.287787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:39.295778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:39.297868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761f7de-e626-40f7-b8ab-be097873482b service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:39.307025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 727/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:40.109702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f29b-2a36-4a06-b0fe-5afc140290e7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:40.110272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f29b-2a36-4a06-b0fe-5afc140290e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:40.110272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480f29b-2a36-4a06-b0fe-5afc140290e7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:40.111663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:40.116570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99f8f-4b71-449d-b230-a17adaab394a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:40.116955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99f8f-4b71-449d-b230-a17adaab394a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:40.117143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b99f8f-4b71-449d-b230-a17adaab394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:40.135198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f9b99f8f-4b71-449d-b230-a17adaab394a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:40.136485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f9b99f8f-4b71-449d-b230-a17adaab394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:40.144922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:40.292151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480f29b-2a36-4a06-b0fe-5afc140290e7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:40.306162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480f29b-2a36-4a06-b0fe-5afc140290e7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:40.311443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:40.312088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:40.312377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:40.313172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:40.321114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:40.322695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2084f50-5f1b-46ba-8737-70b092303073 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:40.332726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 712/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:42.248722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8dd31-61b4-45e7-8517-517a6734b828 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:42.249383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8dd31-61b4-45e7-8517-517a6734b828 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:42.249383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8dd31-61b4-45e7-8517-517a6734b828 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:42.251291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:42.256539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dea101-918e-46b0-bfb4-6fd6a27ab64d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:42.256949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dea101-918e-46b0-bfb4-6fd6a27ab64d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:42.257129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dea101-918e-46b0-bfb4-6fd6a27ab64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:42.274510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-93dea101-918e-46b0-bfb4-6fd6a27ab64d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:42.275728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-93dea101-918e-46b0-bfb4-6fd6a27ab64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:42.283316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:42.321971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8dd31-61b4-45e7-8517-517a6734b828 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:42.332572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8dd31-61b4-45e7-8517-517a6734b828 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:42.336082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:42.336721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:42.337024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:42.337291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:42.346300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:42.347903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8c633-e0ae-4458-8ccf-90a39c90e17e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:42.357518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 730/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:44.533508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e84ea7-9ba4-4e94-a1cb-329c30f8c0e1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:44.534037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e84ea7-9ba4-4e94-a1cb-329c30f8c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:44.534037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e84ea7-9ba4-4e94-a1cb-329c30f8c0e1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:44.535730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:44.540845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d9a-1e09-443c-ba5f-f2eb6a6982c9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:44.541277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d9a-1e09-443c-ba5f-f2eb6a6982c9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:44.541436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d9a-1e09-443c-ba5f-f2eb6a6982c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:44.558738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-118f1d9a-1e09-443c-ba5f-f2eb6a6982c9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:44.559957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-118f1d9a-1e09-443c-ba5f-f2eb6a6982c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:44.567621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:44.605057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e84ea7-9ba4-4e94-a1cb-329c30f8c0e1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:44.615603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e84ea7-9ba4-4e94-a1cb-329c30f8c0e1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:44.619018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:44.619683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:44.619968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:44.620245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:44.629416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:44.630835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879ee79-2a65-492c-be0e-50008faf4efa service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:44.640261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 715/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:46.931741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce98c-172d-4a83-ac99-a24ef8bf2df6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:46.932255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce98c-172d-4a83-ac99-a24ef8bf2df6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:46.932456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbce98c-172d-4a83-ac99-a24ef8bf2df6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:46.933696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:46.939682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe1c2-5630-4883-a359-68d7112320b0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:46.940095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe1c2-5630-4883-a359-68d7112320b0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:46.940254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbe1c2-5630-4883-a359-68d7112320b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:46.958235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0afbe1c2-5630-4883-a359-68d7112320b0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:46.959470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0afbe1c2-5630-4883-a359-68d7112320b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:46.967302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:47.005726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbce98c-172d-4a83-ac99-a24ef8bf2df6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:47.016797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbce98c-172d-4a83-ac99-a24ef8bf2df6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:47.020233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:47.020889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:47.021171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:47.021449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:47.030143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:47.031763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4828d-00d6-4586-81bb-a1e10f34c67c service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:47.041249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 733/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:49.186721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077d92-b6f2-4dfb-925d-ad62fae31d99 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:49.187264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077d92-b6f2-4dfb-925d-ad62fae31d99 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:49.187264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b077d92-b6f2-4dfb-925d-ad62fae31d99 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:49.189152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:49.194297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc6373-a9b3-44cb-8aab-f09331e5971b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:49.194748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc6373-a9b3-44cb-8aab-f09331e5971b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:49.194858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc6373-a9b3-44cb-8aab-f09331e5971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:49.212644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-87bc6373-a9b3-44cb-8aab-f09331e5971b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:49.213824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87bc6373-a9b3-44cb-8aab-f09331e5971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:49.221440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:49.259095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077d92-b6f2-4dfb-925d-ad62fae31d99 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:49.270200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b077d92-b6f2-4dfb-925d-ad62fae31d99 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:49.273664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:49.274303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:49.274587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:49.274867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:49.283621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:49.285310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9653cd-8073-4ddc-bfa1-bb1bdce697dc service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:49.294522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 718/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:51.369215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984317f2-37f2-46a1-9000-9867c77c5cea None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:51.369765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984317f2-37f2-46a1-9000-9867c77c5cea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:51.369765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984317f2-37f2-46a1-9000-9867c77c5cea None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:51.371143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:51.376866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada50e-3aab-475c-9957-c5746c881e92 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:51.377289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada50e-3aab-475c-9957-c5746c881e92 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:51.377435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada50e-3aab-475c-9957-c5746c881e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:51.394822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4eada50e-3aab-475c-9957-c5746c881e92 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:51.396080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4eada50e-3aab-475c-9957-c5746c881e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:51.403868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:51.441698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984317f2-37f2-46a1-9000-9867c77c5cea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:51.452138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984317f2-37f2-46a1-9000-9867c77c5cea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:51.455619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:51.456195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:51.456481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:51.456758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:51.465399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:51.466810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f4626-cd67-4d83-bcc4-af0246cc1694 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:51.476540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 736/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:53.521881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645c15-5abd-40fe-8b03-24f4d3144c19 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:53.522423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645c15-5abd-40fe-8b03-24f4d3144c19 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:53.522423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3645c15-5abd-40fe-8b03-24f4d3144c19 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:53.523799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:53.529038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da35b7c-3cee-4c00-b3b6-636d5a2ad555 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:53.529439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da35b7c-3cee-4c00-b3b6-636d5a2ad555 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:53.529622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da35b7c-3cee-4c00-b3b6-636d5a2ad555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:53.547723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5da35b7c-3cee-4c00-b3b6-636d5a2ad555 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:53.548897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5da35b7c-3cee-4c00-b3b6-636d5a2ad555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:53.556988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:53.594524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3645c15-5abd-40fe-8b03-24f4d3144c19 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:53.605120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3645c15-5abd-40fe-8b03-24f4d3144c19 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:53.608620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:53.609258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:53.609548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:53.609831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:53.618605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:53.620064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045fe97-dc4e-48d9-b16d-51f997cb7491 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:53.629952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 721/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:55.669186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b7ce-5863-4455-b0e4-d62f7be45c93 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:55.669709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b7ce-5863-4455-b0e4-d62f7be45c93 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:55.669709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917b7ce-5863-4455-b0e4-d62f7be45c93 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:55.671292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:55.676646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5295aa-6cff-4a7b-a800-e6638480bdf7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:55.677056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5295aa-6cff-4a7b-a800-e6638480bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:55.677198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5295aa-6cff-4a7b-a800-e6638480bdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:55.694655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-df5295aa-6cff-4a7b-a800-e6638480bdf7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:55.695864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-df5295aa-6cff-4a7b-a800-e6638480bdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:55.704008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:55.740922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b7ce-5863-4455-b0e4-d62f7be45c93 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:55.752411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917b7ce-5863-4455-b0e4-d62f7be45c93 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:55.755988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:55.756633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:55.756925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:55.757213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:55.766583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:55.768390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00426f-3fcf-452a-a62d-e8518bc9a1ce service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:55.777846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 739/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:57.840415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2dad7-112c-4750-868f-937efb352fda None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:57.840978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2dad7-112c-4750-868f-937efb352fda None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:57.840978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2dad7-112c-4750-868f-937efb352fda None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:57.842352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:57.847523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49dbf-6afb-442e-999b-e8bd08dfe3fb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:57.847984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49dbf-6afb-442e-999b-e8bd08dfe3fb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:57.848125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49dbf-6afb-442e-999b-e8bd08dfe3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:57.854196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=79 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:18:57.867264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cfc49dbf-6afb-442e-999b-e8bd08dfe3fb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:18:57.868475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cfc49dbf-6afb-442e-999b-e8bd08dfe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:18:57.876895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:18:57.913161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2dad7-112c-4750-868f-937efb352fda None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:18:57.918457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=73 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:18:57.925458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2dad7-112c-4750-868f-937efb352fda None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:18:57.928976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:18:57.929620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:57.929920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:57.930198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:57.939594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:18:57.941019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831db27-08d3-46ab-833c-7c05e5c69273 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:18:57.950178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 724/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:18:59.990087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb3206-9ac1-4bbb-b351-576fca99673e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:59.990620 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb3206-9ac1-4bbb-b351-576fca99673e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:59.990620 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb3206-9ac1-4bbb-b351-576fca99673e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:18:59.992087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:18:59.997172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b1a2-bcd0-4ca4-8afa-b7878fd4adb4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:18:59.997575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b1a2-bcd0-4ca4-8afa-b7878fd4adb4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:18:59.997732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103b1a2-bcd0-4ca4-8afa-b7878fd4adb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:00.015166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3103b1a2-bcd0-4ca4-8afa-b7878fd4adb4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:00.016915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3103b1a2-bcd0-4ca4-8afa-b7878fd4adb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:00.024818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:00.061495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb3206-9ac1-4bbb-b351-576fca99673e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:00.072205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb3206-9ac1-4bbb-b351-576fca99673e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:00.075721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:00.076360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:00.076650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:00.076931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:00.086204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:00.087681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f41c8-f91c-49b6-8da7-13c67f6a7b92 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:00.097764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 742/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:02.098733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c870-8c7c-4821-88d8-82b60c2abc9b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:02.099300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c870-8c7c-4821-88d8-82b60c2abc9b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:02.099300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28c870-8c7c-4821-88d8-82b60c2abc9b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:02.100660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:02.105814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444c19-2ba0-4738-bf16-165d7b09750d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:02.106197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444c19-2ba0-4738-bf16-165d7b09750d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:02.106376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07444c19-2ba0-4738-bf16-165d7b09750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:02.123689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-07444c19-2ba0-4738-bf16-165d7b09750d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:02.125379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-07444c19-2ba0-4738-bf16-165d7b09750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:02.132963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:02.168580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28c870-8c7c-4821-88d8-82b60c2abc9b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:02.179157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28c870-8c7c-4821-88d8-82b60c2abc9b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:02.182603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:02.183265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:02.183558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:02.183856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:02.193143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:02.194568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf9d40-d804-4b64-aabb-37a00d9a7191 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:02.204411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 727/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:12.118872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=11 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:19:12.199718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=14 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:19:21.671606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835818fe-fdbe-4797-98a2-c189e151a5b4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:21.672175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835818fe-fdbe-4797-98a2-c189e151a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:21.672175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835818fe-fdbe-4797-98a2-c189e151a5b4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:21.673485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 744/1471] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:21.678778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72854ec-ced2-405b-b761-bd4383a14757 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:21.679211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72854ec-ced2-405b-b761-bd4383a14757 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:21.679398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72854ec-ced2-405b-b761-bd4383a14757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:21.697712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a72854ec-ced2-405b-b761-bd4383a14757 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:21.698987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a72854ec-ced2-405b-b761-bd4383a14757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:21.707932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 728/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:21.789737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ecea-9e9b-4838-b34b-5a00df5fe79f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:21.790034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ecea-9e9b-4838-b34b-5a00df5fe79f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:21.790322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59ecea-9e9b-4838-b34b-5a00df5fe79f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:21.791795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 745/1473] 10.222.0.46 () {54 vars in 962 bytes} [Wed Jul 6 15:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:21.796597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbdf78-7667-4116-9884-843dc2aca8f1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:21.797426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbdf78-7667-4116-9884-843dc2aca8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:21.797708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbdf78-7667-4116-9884-843dc2aca8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:21.815855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ffdbdf78-7667-4116-9884-843dc2aca8f1 None None] MFA Rules not processed for user `f11ea126550c4c4a8892522e01ed149a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:21.817032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ffdbdf78-7667-4116-9884-843dc2aca8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:21.823322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 729/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Wed Jul 6 15:19:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:21.830274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59ecea-9e9b-4838-b34b-5a00df5fe79f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:21.840656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59ecea-9e9b-4838-b34b-5a00df5fe79f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:21.844143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:21.844767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:21.845050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:21.845328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:21.854027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:21.855500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe50c2f-74d9-471b-b5f6-f8e04e235988 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:21.865134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 746/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:23.940206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7347c-4939-445d-aae3-4568c9d5c668 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:23.940781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7347c-4939-445d-aae3-4568c9d5c668 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:23.940781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7347c-4939-445d-aae3-4568c9d5c668 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:23.942163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 730/1476] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:23.947242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2a1-4ac4-48c8-b463-39c0fb655683 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:23.947686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2a1-4ac4-48c8-b463-39c0fb655683 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:23.947827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2a1-4ac4-48c8-b463-39c0fb655683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:23.965994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7134d2a1-4ac4-48c8-b463-39c0fb655683 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:23.967191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7134d2a1-4ac4-48c8-b463-39c0fb655683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:23.974996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 747/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:23.998256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7347c-4939-445d-aae3-4568c9d5c668 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:24.011924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7347c-4939-445d-aae3-4568c9d5c668 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:24.016303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:24.016946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:24.017231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:24.017506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:24.018238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:24.020037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:19:24.020037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:19:24.020037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:19:24.021163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c40e5-aff6-4434-b9b9-80d405386c13 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:24.041086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 731/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Wed Jul 6 15:19:23 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:19:26.067507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e4fb-38bf-4656-ba8c-24b743164477 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:26.068087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e4fb-38bf-4656-ba8c-24b743164477 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:26.068087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5e4fb-38bf-4656-ba8c-24b743164477 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:26.069437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 748/1479] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:26.074646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b6384-7346-474f-94d5-06f5f04bcd3e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:26.075084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b6384-7346-474f-94d5-06f5f04bcd3e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:26.075268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b6384-7346-474f-94d5-06f5f04bcd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:26.092730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-631b6384-7346-474f-94d5-06f5f04bcd3e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:26.094418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-631b6384-7346-474f-94d5-06f5f04bcd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:26.102147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 732/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:26.144328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e4fb-38bf-4656-ba8c-24b743164477 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:26.159714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5e4fb-38bf-4656-ba8c-24b743164477 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:26.165282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:26.165917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:26.166203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:26.166479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:26.175356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:26.176924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc599943-36d5-4788-90fe-9b29a984693c service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:26.186090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 749/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:28.366748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dfaca-9b99-45e4-8db7-36b786e25669 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:28.367337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dfaca-9b99-45e4-8db7-36b786e25669 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:28.367454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dfaca-9b99-45e4-8db7-36b786e25669 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:28.368828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 733/1482] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:28.373892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e875beb-7422-4ee2-a10b-0209fa682a50 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:28.374318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e875beb-7422-4ee2-a10b-0209fa682a50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:28.374478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e875beb-7422-4ee2-a10b-0209fa682a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:28.393021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4e875beb-7422-4ee2-a10b-0209fa682a50 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:28.394778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4e875beb-7422-4ee2-a10b-0209fa682a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:28.402831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 750/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:19:28.440378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dfaca-9b99-45e4-8db7-36b786e25669 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:28.451134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dfaca-9b99-45e4-8db7-36b786e25669 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:28.454797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:28.455513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:28.455786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:28.456067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:28.464893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:28.466323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d2381b-73d2-43f5-8a97-5f05a5bd9472 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:28.476311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 734/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:30.530345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881586-c4cb-4f48-ab82-576be762c1ed None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:30.530860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881586-c4cb-4f48-ab82-576be762c1ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:30.530860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881586-c4cb-4f48-ab82-576be762c1ed None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:30.532284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 751/1485] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:30.537703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd14e-bcef-4cc7-b849-d8bc61151c91 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:30.538103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd14e-bcef-4cc7-b849-d8bc61151c91 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:30.538267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbd14e-bcef-4cc7-b849-d8bc61151c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:30.555698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-81cbd14e-bcef-4cc7-b849-d8bc61151c91 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:30.556889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-81cbd14e-bcef-4cc7-b849-d8bc61151c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:30.565138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 735/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:19:30.603317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881586-c4cb-4f48-ab82-576be762c1ed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:30.613925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881586-c4cb-4f48-ab82-576be762c1ed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:30.617439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:30.618081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:30.618371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:30.618650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:30.628069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:30.629547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710dbea-973b-4d72-9c59-2e931ca998c1 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:30.638977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 752/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:32.807090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ce5-1301-42fb-8078-57e91931ab9a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:32.807650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ce5-1301-42fb-8078-57e91931ab9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:32.807650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81ce5-1301-42fb-8078-57e91931ab9a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:32.809003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 736/1488] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:32.814935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8cb2-c661-428a-bfca-3560cfcf025d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:32.815365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8cb2-c661-428a-bfca-3560cfcf025d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:32.815582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee8cb2-c661-428a-bfca-3560cfcf025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:32.833775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a9ee8cb2-c661-428a-bfca-3560cfcf025d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:32.834985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a9ee8cb2-c661-428a-bfca-3560cfcf025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:32.842746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 753/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:32.882124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81ce5-1301-42fb-8078-57e91931ab9a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:32.893167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c81ce5-1301-42fb-8078-57e91931ab9a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:32.897025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:32.897663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:32.897947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:32.898235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:32.907003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:32.908470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6a2e7-6510-4e4d-9569-07860365cf4c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:32.918130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 737/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:35.169793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2047d5-93f0-4545-9a4b-92394ca7e224 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:35.170332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2047d5-93f0-4545-9a4b-92394ca7e224 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:35.170332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2047d5-93f0-4545-9a4b-92394ca7e224 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:35.172606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 754/1491] 10.222.0.46 () {54 vars in 926 bytes} [Wed Jul 6 15:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:19:35.177661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89efb0a-bd48-4978-b022-43ed4228cb53 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:35.178111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89efb0a-bd48-4978-b022-43ed4228cb53 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:35.178243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89efb0a-bd48-4978-b022-43ed4228cb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:35.195710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b89efb0a-bd48-4978-b022-43ed4228cb53 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:19:35.196913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b89efb0a-bd48-4978-b022-43ed4228cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:19:35.204732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 738/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Wed Jul 6 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:19:35.244494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2047d5-93f0-4545-9a4b-92394ca7e224 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:19:35.256535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2047d5-93f0-4545-9a4b-92394ca7e224 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:19:35.260296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:19:35.260951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:19:35.261241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:19:35.261520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:19:35.271590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:19:35.273023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4536-cf1f-4426-9e3f-3a2c52524468 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:19:35.283564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 755/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:19:45.190668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=33 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:19:45.278342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=33 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:21:01.512753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.513334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.513334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.531982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:21:01.533172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:01.541229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 739/1494] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:21:01.548907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.549366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.549471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.567422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:21:01.568625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:01.577007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 756/1495] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:21:01.581604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.591171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23b618-67b9-449f-9e41-2940a6575d13 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.596345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.596999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.597289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.597567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.598294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:21:01.598607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.599650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.599650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:21:01.599650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:21:01.600525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.611922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 740/1496] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:21:01.619677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.628806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d8885a-9233-4149-be98-e4d7ef6ef162 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.633322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.633997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.634285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.634565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.635296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.637075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.637075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:21:01.637075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:21:01.638256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.674437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 757/1497] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:21:01 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:21:01.678518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.688221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56e44d-8590-4dc7-825d-ab99838c72b1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.744953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.745635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.745928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.746211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.746905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.747974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.747974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:21:01.747974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:21:01.749208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.769670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 741/1498] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:21:01 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:21:01.773659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.784542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1917df6-8239-4d64-b6dc-c5d579501079 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.789275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.789938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.790228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.790508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.791261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.792274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.792274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:21:01.792274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:21:01.793150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.805677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 758/1499] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:21:01.809639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.819354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b13227-fdc3-48b5-b319-5ace540f7a31 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.823821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.824487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.824780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.825062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.825797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.827356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.827356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:21:01.827356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:21:01.828233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e1e4e-baf3-4f85-bdd6-399d4780bb84 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.840632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 742/1500] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:21:01.845081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:01.854683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78d504-b75a-44d1-a572-4d132b263f21 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:01.859969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:01.860870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.861155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.861435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] PATH_INFO: `/v3/projects/97522bfff3194924a43d8841b06b2d7d/users/e95c323472d542a9a747d78ceab1fb7f/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.876307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97522bfff3194924a43d8841b06b2d7d, user_id=e95c323472d542a9a747d78ceab1fb7f, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:01.877321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:01.877321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:21:01.877321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:21:01.879242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dfc81-e821-4d97-9800-9dac528ef945 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:01.892397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 759/1501] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:21:01 2022] PUT /identity/v3/projects/97522bfff3194924a43d8841b06b2d7d/users/e95c323472d542a9a747d78ceab1fb7f/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:21:01.904306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca7fca-4f2b-42a7-9c0d-5312d7790ac2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:01.904666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca7fca-4f2b-42a7-9c0d-5312d7790ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:01.904958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca7fca-4f2b-42a7-9c0d-5312d7790ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:01.931699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-33ca7fca-4f2b-42a7-9c0d-5312d7790ac2 None None] MFA Rules not processed for user `e95c323472d542a9a747d78ceab1fb7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:21:01.932908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33ca7fca-4f2b-42a7-9c0d-5312d7790ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:01.994903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 743/1502] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:21:02.008251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc848a9-5e2f-4ffb-b9b6-273a8af3e99d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.008578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc848a9-5e2f-4ffb-b9b6-273a8af3e99d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.008817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc848a9-5e2f-4ffb-b9b6-273a8af3e99d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.010254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 760/1503] 10.222.0.46 () {54 vars in 963 bytes} [Wed Jul 6 15:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:21:02.016762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7db7-2097-408b-b918-e225551ce9c5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.017058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7db7-2097-408b-b918-e225551ce9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.017502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7db7-2097-408b-b918-e225551ce9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.036759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0bbd7db7-2097-408b-b918-e225551ce9c5 None None] MFA Rules not processed for user `edbb85bda10d45bf8f66c6ee0561c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:21:02.037909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0bbd7db7-2097-408b-b918-e225551ce9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:02.080662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:21:02.088904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 744/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Wed Jul 6 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:21:02.096574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc848a9-5e2f-4ffb-b9b6-273a8af3e99d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:02.106858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc848a9-5e2f-4ffb-b9b6-273a8af3e99d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:02.110279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:02.110910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.111193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.111556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.119599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:02.121003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:02.130082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 761/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:21:02.171374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd4c2-6129-4f17-92db-5997c848b2d5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.171668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd4c2-6129-4f17-92db-5997c848b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.171951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbd4c2-6129-4f17-92db-5997c848b2d5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.173244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 745/1506] 10.222.0.46 () {52 vars in 829 bytes} [Wed Jul 6 15:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:21:02.177047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:02.237148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156fb3e-fc00-4d37-8974-661dd3380ade service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:02.241806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:02.242685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.243565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.243895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] PATH_INFO: `/v3/users/e95c323472d542a9a747d78ceab1fb7f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.245250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95c323472d542a9a747d78ceab1fb7f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:02.246286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:02.246286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:21:02.246286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:21:02.247522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:02.311782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95c323472d542a9a747d78ceab1fb7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.329537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95c323472d542a9a747d78ceab1fb7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.338559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95c323472d542a9a747d78ceab1fb7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.347052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:21:02.351876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95c323472d542a9a747d78ceab1fb7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.352544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95c323472d542a9a747d78ceab1fb7f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.359854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 762/1507] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:21:02 2022] DELETE /identity/v3/users/e95c323472d542a9a747d78ceab1fb7f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:21:02.364406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.364700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.365007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.382406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:21:02.383640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:02.440821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 746/1508] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:21:02.450076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:02.451545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:02.507445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaa1f2-1026-453c-92dc-ec63f54eed7d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:02.511012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:02.511700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.512075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.512355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.521198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:02.522637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837c45d-f040-4aa3-9974-70ca29c63e44 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:02.532655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 763/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:21:02.882279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:21:02.883837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:21:02.895474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b58ab-5b62-47cd-bbb6-a53b4530f954 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:21:02.900359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:21:02.901254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:21:02.901539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:21:02.901818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] PATH_INFO: `/v3/projects/97522bfff3194924a43d8841b06b2d7d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:21:02.903126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97522bfff3194924a43d8841b06b2d7d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:21:02.904066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:21:02.904066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:21:02.904066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:21:02.905792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:21:02.967795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97522bfff3194924a43d8841b06b2d7d', 'request_id': 'req-e93dae5f-8174-4bfc-a90d-f07998a48b3a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.977421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97522bfff3194924a43d8841b06b2d7d', 'request_id': 'req-e93dae5f-8174-4bfc-a90d-f07998a48b3a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:02.991409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97522bfff3194924a43d8841b06b2d7d', 'request_id': 'req-e93dae5f-8174-4bfc-a90d-f07998a48b3a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:03.002319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] The token cache is being invalidate because project 97522bfff3194924a43d8841b06b2d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:21:03.002609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:03.003297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e93dae5f-8174-4bfc-a90d-f07998a48b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:21:03.004949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 747/1510] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:21:02 2022] DELETE /identity/v3/projects/97522bfff3194924a43d8841b06b2d7d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:21:12.528020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=29 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:21:12.999328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:22:37.545369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:37.545939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:37.545939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:37.547849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:37.552682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104a8-d111-41ba-9cea-b8defa5a3ec0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:37.553085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104a8-d111-41ba-9cea-b8defa5a3ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:37.553246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1104a8-d111-41ba-9cea-b8defa5a3ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:37.571171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3a1104a8-d111-41ba-9cea-b8defa5a3ec0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:37.572390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3a1104a8-d111-41ba-9cea-b8defa5a3ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:37.628939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:37.779383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:37.781665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:37.853915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98debbef-2dfc-4a2d-87d4-55020f98c5e3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:37.859692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:37.860342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:37.860625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:37.860904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:37.868814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:37.870417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed5998-47e3-4c09-a2ed-3b971052f8a4 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:37.879690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 765/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:38.716209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a94f1-3aec-411a-bcbb-676d452f3967 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:38.716753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a94f1-3aec-411a-bcbb-676d452f3967 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:38.716753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a94f1-3aec-411a-bcbb-676d452f3967 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:38.718122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:38.722812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847442-8714-483f-a3db-9791173a4410 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:38.723229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847442-8714-483f-a3db-9791173a4410 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:38.723411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f847442-8714-483f-a3db-9791173a4410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:38.741064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8f847442-8714-483f-a3db-9791173a4410 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:38.742267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f847442-8714-483f-a3db-9791173a4410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:38.750483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:38.901117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a94f1-3aec-411a-bcbb-676d452f3967 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:38.915243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a94f1-3aec-411a-bcbb-676d452f3967 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:38.920476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:38.921115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:38.921884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:38.922174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:38.930129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:38.931784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230836f-031d-49ae-af50-47fd178c5f13 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:38.941562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 750/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:39.513796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ef9e-0595-4a56-9064-f38337ec029b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:39.514005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ef9e-0595-4a56-9064-f38337ec029b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:39.514279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661ef9e-0595-4a56-9064-f38337ec029b None None] PATH_INFO: `/v3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:39.515668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 767/1517] 10.222.0.16 () {54 vars in 925 bytes} [Wed Jul 6 15:22:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:22:39.521782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd763-ca3c-4d4b-8c08-265b394fd9e9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:39.522082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd763-ca3c-4d4b-8c08-265b394fd9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:39.522368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bd763-ca3c-4d4b-8c08-265b394fd9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:39.540866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0c3bd763-ca3c-4d4b-8c08-265b394fd9e9 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:39.542087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c3bd763-ca3c-4d4b-8c08-265b394fd9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:39.604034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 751/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Jul 6 15:22:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:43.340992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e01b-d28f-4fe9-be6d-e9c9f5cc0a79 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:43.341457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e01b-d28f-4fe9-be6d-e9c9f5cc0a79 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:43.341552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404e01b-d28f-4fe9-be6d-e9c9f5cc0a79 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:43.342928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 768/1519] 10.222.0.46 () {54 vars in 960 bytes} [Wed Jul 6 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:43.348703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25244473-cf66-4d01-ac84-d5a1145a01d5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:43.349143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25244473-cf66-4d01-ac84-d5a1145a01d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:43.349241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25244473-cf66-4d01-ac84-d5a1145a01d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:43.367302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-25244473-cf66-4d01-ac84-d5a1145a01d5 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:43.368486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-25244473-cf66-4d01-ac84-d5a1145a01d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:43.376888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 752/1520] 10.222.0.46 () {58 vars in 1057 bytes} [Wed Jul 6 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:43.384446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404e01b-d28f-4fe9-be6d-e9c9f5cc0a79 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:43.397467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404e01b-d28f-4fe9-be6d-e9c9f5cc0a79 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:43.402299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:43.402931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:43.403239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:43.403530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:43.412322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:43.413915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8130f-203c-49ca-b8a7-184455c6009b service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:43.423956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 769/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:50.166737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88eb8d3-f199-486d-9919-73043b5b1f1b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:50.167363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88eb8d3-f199-486d-9919-73043b5b1f1b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:50.167363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88eb8d3-f199-486d-9919-73043b5b1f1b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:50.168704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:50.174006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696e1e8-399a-4a36-bfe0-6ce902a5e3f6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:50.174439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696e1e8-399a-4a36-bfe0-6ce902a5e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:50.174572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696e1e8-399a-4a36-bfe0-6ce902a5e3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:50.192926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b696e1e8-399a-4a36-bfe0-6ce902a5e3f6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:50.194124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b696e1e8-399a-4a36-bfe0-6ce902a5e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:50.201927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:50.352585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88eb8d3-f199-486d-9919-73043b5b1f1b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:50.367410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88eb8d3-f199-486d-9919-73043b5b1f1b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:50.372679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:50.373319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:50.373606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:50.373885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:50.382460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:50.384082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715339e1-b9cf-41e0-8c6a-d1511280e1bd service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:50.393328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 754/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:51.208234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877485e-4494-4360-bac5-686feb70d9f2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:51.208821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877485e-4494-4360-bac5-686feb70d9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:51.208821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877485e-4494-4360-bac5-686feb70d9f2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:51.210190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 15:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:51.215047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5d4f-3947-46cb-b61e-72b74b20dc5e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:51.215522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5d4f-3947-46cb-b61e-72b74b20dc5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:51.215651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd5d4f-3947-46cb-b61e-72b74b20dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:51.233893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-34cd5d4f-3947-46cb-b61e-72b74b20dc5e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:51.235153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-34cd5d4f-3947-46cb-b61e-72b74b20dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:51.243387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:51.392828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877485e-4494-4360-bac5-686feb70d9f2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:51.407595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877485e-4494-4360-bac5-686feb70d9f2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:51.413142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:51.413787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:51.414079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:51.414368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:51.422950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:51.424584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8053fd9-3447-4b86-a80d-5205de16844f service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:51.433836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 772/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:55.364735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:55.365235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:55.365494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:55.366903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:55.372170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb61a-1808-4e0f-b70c-040eebdffb41 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:55.372576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb61a-1808-4e0f-b70c-040eebdffb41 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:55.372727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deb61a-1808-4e0f-b70c-040eebdffb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:55.393428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99deb61a-1808-4e0f-b70c-040eebdffb41 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:55.394618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99deb61a-1808-4e0f-b70c-040eebdffb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:55.402298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:55.473668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:55.475421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:55.531883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17e76e-4f3a-44dc-8fc7-25eb5365f16d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:55.535708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:55.536354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:55.536648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:55.536930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:55.545699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:55.547689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a255e-4893-4659-b130-65bb57d2438f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:55.552623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:22:55.557881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 757/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:22:57.561781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c36c-84ef-447d-a28e-56f7b05b01f9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:57.562288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c36c-84ef-447d-a28e-56f7b05b01f9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:57.562384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8c36c-84ef-447d-a28e-56f7b05b01f9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:57.563800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:22:57.568760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56ec4-8a94-4ce2-8642-157af5f01ab1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:57.569172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56ec4-8a94-4ce2-8642-157af5f01ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:57.569322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f56ec4-8a94-4ce2-8642-157af5f01ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:57.586722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-60f56ec4-8a94-4ce2-8642-157af5f01ab1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:22:57.587965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-60f56ec4-8a94-4ce2-8642-157af5f01ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:22:57.596163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:22:57.630587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c36c-84ef-447d-a28e-56f7b05b01f9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:22:57.641094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8c36c-84ef-447d-a28e-56f7b05b01f9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:22:57.644542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:22:57.645179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:22:57.645466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:22:57.645745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:22:57.654383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:22:57.656339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf86a9a-e8f0-4a63-9b20-4c0d4ee7121b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:22:57.665731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 775/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:23:01.159387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:01.159987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:01.159987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:01.161581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:23:01.166619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56c30-497b-4e54-963f-2ce106760940 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:01.167039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56c30-497b-4e54-963f-2ce106760940 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:01.167156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f56c30-497b-4e54-963f-2ce106760940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:01.304102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-25f56c30-497b-4e54-963f-2ce106760940 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:23:01.305227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-25f56c30-497b-4e54-963f-2ce106760940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:23:01.313337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:23:01.613022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:23:01.614690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:23:01.672407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692ef36-be47-453f-85aa-ee342add7135 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:23:01.675946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:23:01.676587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:01.676874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:01.677153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:01.685820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:23:01.687891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb129-3210-4b8e-8168-5382ee96870e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:23:01.697031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 760/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:23:03.704415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:03.705015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:03.705015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:03.706337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 15:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:23:03.711551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813424d4-24f8-41d0-aae9-7685116e3a36 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:03.711937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813424d4-24f8-41d0-aae9-7685116e3a36 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:03.712111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813424d4-24f8-41d0-aae9-7685116e3a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:03.730470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-813424d4-24f8-41d0-aae9-7685116e3a36 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:23:03.731702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-813424d4-24f8-41d0-aae9-7685116e3a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:23:03.739453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 15:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:23:04.040362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:23:04.042010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:23:04.049565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=65 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:23:04.056013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b1185-4134-4c42-a173-f7d43d73c423 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:23:04.059543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:23:04.060189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:23:04.060477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:23:04.060760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:23:04.070188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:23:04.071644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23064f68-58db-4889-911c-10810c1c52ae service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:23:04.081642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 778/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:23:13.725259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=32 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:23:14.076834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:31.713215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be01a-c5c6-4114-a321-1a6a42f7e48b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.713862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be01a-c5c6-4114-a321-1a6a42f7e48b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.713862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3be01a-c5c6-4114-a321-1a6a42f7e48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.738574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ab3be01a-c5c6-4114-a321-1a6a42f7e48b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:31.739830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab3be01a-c5c6-4114-a321-1a6a42f7e48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:31.753930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 762/1540] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:31.755011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4d04-69d5-43c1-ae66-a512e26145eb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.755916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4d04-69d5-43c1-ae66-a512e26145eb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.756411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d4d04-69d5-43c1-ae66-a512e26145eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.765544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.765834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.766116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.785099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:31.786308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:31.792392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c7d4d04-69d5-43c1-ae66-a512e26145eb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:31.794193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c7d4d04-69d5-43c1-ae66-a512e26145eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:31.802540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 763/1541] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:31.807465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:31.814498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 779/1542] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:31.822478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f6d3e-839b-4d7e-aed2-a08a6baf3e7e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:31.823194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.823523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.823806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.831300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:31.831992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.832278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.832555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.833302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:31.833610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:31.839246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:31.863392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 764/1543] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:31.868193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:31.879200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:31.880622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:31.886468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076f25-7c65-428b-b326-9e85f273ee64 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:31.897140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:31.897844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.898134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.898410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.899133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:31.900370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:31.900370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:24:31.900370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:24:31.901573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:31.905496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 780/1544] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:31.910939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:31.928075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1018ec-3930-4e9b-bc55-3e5adb6f7d39 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:31.945544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:31.946493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:31.946790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:31.947076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:31.947868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:31.948185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:31.949268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:31.949268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:24:31.949268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:24:31.950179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:31.957869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 765/1545] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:31.962663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:31.980510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 781/1546] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:31.989666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.010539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a9d1-cb07-41e1-9f2f-558f194925b2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.059780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b31210-e358-4f36-a7ce-26df5ead4167 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.084805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.085821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.085821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.086407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.087160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.089372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.121367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.122502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.123027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.123560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.124847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.126417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 766/1547] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:24:32.127412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.135468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.153351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf8286-965a-4041-a65f-9e582170b982 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.173189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.173903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.174194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.174473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.175385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.176982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaa1e8-1a71-488d-a59e-7e16fcfcc9dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.188434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 767/1548] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:24:32.189824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.190126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.190455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.192644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 782/1549] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:31 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:24:32.193789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.217786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.219000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.327135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 768/1550] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.328097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.340758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f9b4-1085-407c-b266-855ec4b9a105 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.348543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606392d0-961a-4f24-b6fb-3c2956b8af7c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.351718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.352396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.352682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.352959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.353656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.355618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84377cb1-eacc-48d8-8d70-653f1bdcae48 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.370813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.371812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.372347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.372637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.373624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.379589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf9f17-3b4c-4f50-a434-e16e2f89ad8d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.391632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 769/1551] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/users => generated 286 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:32.396685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 783/1552] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:32.397089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.397433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.397723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.397907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.398172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.398641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.420745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.422328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.428574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.429756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.439245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 770/1553] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.439245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.445547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 784/1554] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.446668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.447046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a71614-d4d2-44df-bfd5-09ec6435db3f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.452260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.452970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.453262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.453721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.454502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.456821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0bebd-6b03-4eb6-925e-7b56deee75b1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.460558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3edf83-0a1c-4419-a77f-a96714b55e2e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.462795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:32.473321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.473321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.473618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.473906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] PATH_INFO: `/v3/projects/8f95961717a844969240b7b718c8e764/users/30c6a87965c349a3aeebeddf65a4c0fa/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.474208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 771/1555] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:32.475553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e7f4-4291-47f1-bdd3-820ad6427092 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.475841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e7f4-4291-47f1-bdd3-820ad6427092 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.476123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e7f4-4291-47f1-bdd3-820ad6427092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.492495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f95961717a844969240b7b718c8e764, user_id=30c6a87965c349a3aeebeddf65a4c0fa, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.495205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379fea0-0a9b-4bac-9854-19b97cca25f4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.497379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5da7e7f4-4291-47f1-bdd3-820ad6427092 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.498600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5da7e7f4-4291-47f1-bdd3-820ad6427092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.508284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 772/1556] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.508834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 785/1557] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:32 2022] PUT /identity/v3/projects/8f95961717a844969240b7b718c8e764/users/30c6a87965c349a3aeebeddf65a4c0fa/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:32.510156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.510377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.510496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.510716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.511119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.511594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.532449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.532672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.533916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.534575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.558919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:32.598105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 773/1558] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.598885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.599347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 786/1559] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.600242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.609429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2174e-9527-4d4f-b740-373853c78974 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.614504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6668-f65f-43e8-bd44-642a9a04f823 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.614630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.615306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.615598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.615885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.616770 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.618376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f1309-eb99-40a3-8bd8-d5c25e3461e6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.619116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.619810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.620096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.620381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.621109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:32.621421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.623015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d96c0-0d5d-4d3e-abec-b2e2c34b212c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.628494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 787/1560] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:32.629705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e60d1-aa29-401b-a696-bb748f59332b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.629998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e60d1-aa29-401b-a696-bb748f59332b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.630281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e60d1-aa29-401b-a696-bb748f59332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.637121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 774/1561] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:32.639546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e117a-834f-4559-a097-03f115dff7b7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.639846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e117a-834f-4559-a097-03f115dff7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.640158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e117a-834f-4559-a097-03f115dff7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.648107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ac6e60d1-aa29-401b-a696-bb748f59332b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.649322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ac6e60d1-aa29-401b-a696-bb748f59332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.657225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 788/1562] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.660108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-526e117a-834f-4559-a097-03f115dff7b7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.660754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.661050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.661366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-526e117a-834f-4559-a097-03f115dff7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.661479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.670257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 775/1563] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.677040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248655-8a3c-4814-825f-315679b14a0f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.677040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248655-8a3c-4814-825f-315679b14a0f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.677040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248655-8a3c-4814-825f-315679b14a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.681041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.682242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.690059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 789/1564] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.690894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.701604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-86248655-8a3c-4814-825f-315679b14a0f None None] MFA Rules not processed for user `30c6a87965c349a3aeebeddf65a4c0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.702698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536eac2-cfbe-446d-8e3f-c62e230b0e4c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.702821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-86248655-8a3c-4814-825f-315679b14a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.707267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.707928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.708211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.708488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.709218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:32.709557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.711135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1686c3-a0a2-4d75-af1f-a68ae3053166 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.724669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 790/1565] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:32.726615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.726915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.727201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.744984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.746176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.754337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 791/1566] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.755088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.764817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd7a31-1db2-4718-9d9b-f8c5129e0858 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.769443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 776/1567] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.770094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.770300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86248655-8a3c-4814-825f-315679b14a0f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.771063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.771376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.771653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] PATH_INFO: `/v3/projects/2547c48c6bbd415aae27ed306f22ebbc/users/df7bdd442f0f4355bb75c64409fff6bb/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.781232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86248655-8a3c-4814-825f-315679b14a0f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.786808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.787546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.787796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.788080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.788498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2547c48c6bbd415aae27ed306f22ebbc, user_id=df7bdd442f0f4355bb75c64409fff6bb, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.788782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.790724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1419c-089e-4da0-b38f-78a3dfb9d811 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.791170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f230cc9-8324-47c9-a435-15fef0f6a85a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.815479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 792/1568] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:32 2022] PUT /identity/v3/projects/2547c48c6bbd415aae27ed306f22ebbc/users/df7bdd442f0f4355bb75c64409fff6bb/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:32.815479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.815479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.815479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.833697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 777/1569] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:32.835001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.835448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.835678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.838894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.840133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.854108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:32.855412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:32.887337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:32.900754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 793/1570] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.905400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.915587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2f588-2c68-4764-84c2-2391d96eac50 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.919483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 778/1571] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:32.920579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.923583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.924255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.924545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.924856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.925601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:32.925918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.927557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.930896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ffecd-7405-48b5-9020-73f6c097e9cf None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.935630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.936281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.936569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.936849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.937593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.939183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 794/1572] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:32.939507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38595f92-ffce-47cf-a546-12bb6dd5929a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.940063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.950631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd89fe0-8879-405b-a51c-9cf45f311946 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:32.955766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:32.957032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.957353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.957636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.958419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:32.958728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:32.960368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:32.966529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 779/1573] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:32.968071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96930-ade4-4cc2-acc0-332702df1d3c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.968201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96930-ade4-4cc2-acc0-332702df1d3c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.968424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e96930-ade4-4cc2-acc0-332702df1d3c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.970075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 780/1574] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 6 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:32.970772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:32.971121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:32.971433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:32.973948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 795/1575] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:32.974776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:32.984801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f65c7-ef9f-4791-a22f-409827049f5b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.000935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] MFA Rules not processed for user `df7bdd442f0f4355bb75c64409fff6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:33.002157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:33.016234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.043758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:24:33.053831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:24:33.055520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.070428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 796/1576] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.071278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.071532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 781/1577] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:33.072803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.081338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847e49-c339-4091-8233-ce3df8b3f8a7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.083531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994f9c1-541a-4ca3-b98f-0b53bb2e4930 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.085935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.086643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.086937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.087235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.088062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:33.088284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.088399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.088939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.089232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.089516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.089959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.090246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:33.090567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.092181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.102164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 797/1578] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:33.102843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.107403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 782/1579] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:33.107403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.114804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721b560-fefb-4cc0-a96d-06e82fb155c0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.118081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c4080-a18b-4995-ac0f-1c581acad392 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.119492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.120147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.120437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.120719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.121410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.122816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.123332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3d5c4-dc0a-41ed-82e1-e9e4fc954560 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.123494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.123779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.124059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.124744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.126667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.147676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 783/1580] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 282 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.148459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.152831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 798/1581] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.152831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.152831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.152831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.161231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831d0f4-186d-434c-bb52-3bf2f7d1a51e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.170566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:33.171790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:33.220181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.221316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.222153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.222442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.223146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.225093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.238910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 799/1582] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:33.239797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.249362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.255821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce16a2-6a74-45dc-88cd-0123003108c6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.259376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 784/1583] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.260784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.315610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.316200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.316485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.316758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.317490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.319102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.321661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650d9e0-ec19-4018-9650-5844f5e08657 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.326173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.326849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.327133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.327442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.328131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.328749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 800/1584] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.330059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.332508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.341832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46ffb9-2608-474b-9d8b-8fd8e42373a1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.347280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.348027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.348320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.348551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.349329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.351276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.356871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 785/1585] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.357771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.361784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.381684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 801/1586] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.382768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.420668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4364d1-1a57-44b7-95ec-019d456032cb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.445613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6dfe3-fb3c-4859-8e87-4c82edd1648d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.450149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.450925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.451211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.451525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.452285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.453891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.463566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 802/1587] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.464348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.474587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9629016-77fa-475c-b3f5-946ce110d3e7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.479080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.479754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.480074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.480305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.480986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.481358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.482030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.482345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.482622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.482856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.483346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.485246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.503469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 803/1588] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.507163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.512712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 786/1589] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.513500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.537015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.580301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a1cc-c824-4f91-8461-204e735e27a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.580514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e16cf-9cde-43d6-aba6-623e567fcbb2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.585626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.586306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.586599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.586885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.587639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.589560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.610179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 787/1590] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.611439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.620531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9aca0-7f57-4def-b9f2-9d324c17d7cb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.625147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.625799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.626090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.626372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.627099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.628746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.641505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 788/1591] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.641505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.644192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.644859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.645151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.645435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.648858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175c07-4690-4ed4-abab-1a57bb9463f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.649660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.653768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.654525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.654973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.655609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.656765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.659143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.679612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 789/1592] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 291 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.680571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.690382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276d38-b09a-4f8b-aba9-96fe94e810a5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.694976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.695673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.695956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.696238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.696913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.698975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.704371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.706511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.716905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 804/1593] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:33.717573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.719405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 790/1594] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 293 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.724091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.728260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d15d2-6036-4f18-9081-5568f8422367 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.733006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.733694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.733981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33156f-247b-4464-81e9-802f373a3c3a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.734093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.734259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.734991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.736811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.738707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.739391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.739676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.739956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.740711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.742392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.747619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 805/1595] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.751470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.755933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 791/1596] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.755933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.761507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32577e-6447-4668-b10c-e8c2fe07f90d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.766481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.766721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db238fb-35c3-4616-96b8-0de3a74a2593 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.767477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.767838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.768188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.769470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.771625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.772232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.772966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.773270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.773550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.774329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.776030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.788284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 792/1597] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.792719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.795094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 806/1598] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:33 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:33.795852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.805711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f85a1-3314-4e01-acc8-57b7cdedda8f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.806656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f641b-fdcb-4758-8ecb-f2fe8e2c9088 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.810579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.811346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.811613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.811745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.812096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.812734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.812958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.813154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.813509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] PATH_INFO: `/v3/projects/6707399c704e4c97939dc84df58cb39f/users/0f5497ff570f4affbbd1b2f9129c24ac/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.814745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.826604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 807/1599] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.828606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.829197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6707399c704e4c97939dc84df58cb39f, user_id=0f5497ff570f4affbbd1b2f9129c24ac, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.830370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:33.830370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:24:33.830370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:24:33.833094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.840478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0335f-496d-4839-84f2-dbaf6caa2068 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.849791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 793/1600] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:33 2022] PUT /identity/v3/projects/6707399c704e4c97939dc84df58cb39f/users/0f5497ff570f4affbbd1b2f9129c24ac/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:33.850604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.903181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.904060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.904757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.904757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.905456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.907065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.923615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 808/1601] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.924982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.932402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedb683-72fc-45c8-a478-c064eb7a19dd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.936123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ef680-1e93-42cc-a1cd-b0bac397caec admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.937289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.938225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.938539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.938819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] PATH_INFO: `/v3/projects/3bd8f212edb44e65bdf22e130b50cc01/users/192a3958684e42328536ebff80e351d4/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.941264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.941935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.942239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.942524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.943344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.944997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.960479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 809/1602] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:33.960479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:33.960993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd8f212edb44e65bdf22e130b50cc01, user_id=192a3958684e42328536ebff80e351d4, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.963825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.969785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12dcdd-eaab-49d7-9be5-08d33afb1d02 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:33.974670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:33.976545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:33.977474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:33.977775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:33.978073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:33.979008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:33.980638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:33.980982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 794/1603] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:33 2022] PUT /identity/v3/projects/3bd8f212edb44e65bdf22e130b50cc01/users/192a3958684e42328536ebff80e351d4/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:33.981935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.001560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 810/1604] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:34.003239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.015354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50198d-85e9-4f42-b63f-4e51b5e335e3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.049767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c4bd-532d-44e2-8ab9-79d67315ef39 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.054433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.055104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.055415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.055691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.056445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.057941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:34.058090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ee11d-2274-4519-a873-5f2fafdcbcec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.070335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 795/1605] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:34.071730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.072007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.072291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.083136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.084122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.084516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.084912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] PATH_INFO: `/v3/projects/b1040826386b48b9ac0974c0002cec9f/users/8126a3e25e7e4df29f75b76d8f131e36/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.090247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.091481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.099988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 796/1606] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.100726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.103942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1040826386b48b9ac0974c0002cec9f, user_id=8126a3e25e7e4df29f75b76d8f131e36, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.106744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.110517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab50c-4fcc-47e6-80ee-7ce412c842c7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.116407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.117404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.117697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.117979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.118731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.120416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f07e6-677e-4a68-8233-6ce24b171231 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.122447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 811/1607] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/b1040826386b48b9ac0974c0002cec9f/users/8126a3e25e7e4df29f75b76d8f131e36/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.123494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.134572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 797/1608] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:34.136160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.136563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.136856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.164961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] MFA Rules not processed for user `192a3958684e42328536ebff80e351d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.166184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.201340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670aa93-4a96-413e-88c6-162161ccb4ab admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.207057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.208128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.208427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.208712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] PATH_INFO: `/v3/projects/9be64cd89e3b46738dca85e516a60d8a/users/213ac803c2cb4cbfb4d73ec4fb07d552/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.224397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be64cd89e3b46738dca85e516a60d8a, user_id=213ac803c2cb4cbfb4d73ec4fb07d552, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.227098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.241905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 812/1609] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/9be64cd89e3b46738dca85e516a60d8a/users/213ac803c2cb4cbfb4d73ec4fb07d552/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.242506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.257691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 798/1610] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.258693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.269957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b7e7-1677-4f12-82cb-875cd374b2dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.313982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d939669-f1a3-476d-8bc3-55e8529fc79d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.319137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.320059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.320347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.320621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] PATH_INFO: `/v3/projects/ef44ecc951964d51925463c5e09f9fbb/users/46896be59b73481287235aac3d630f4e/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.325515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.326222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.326513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.326794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.327582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.329176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.336222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef44ecc951964d51925463c5e09f9fbb, user_id=46896be59b73481287235aac3d630f4e, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.338748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.339082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 799/1611] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:34.339921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.344609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:34.353343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 813/1612] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/ef44ecc951964d51925463c5e09f9fbb/users/46896be59b73481287235aac3d630f4e/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.354150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.388649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:34.404221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d9068-6db7-47cb-a33d-3d3ec40b6643 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.422376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e9c0-30af-4be5-9b17-12986401b432 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.426910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.427603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.427913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.428189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.428943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.430516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.440347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 814/1613] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:34.440908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.449915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b500b80-ed55-4d0f-96bb-8376fb64aec5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.454317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.455342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.455622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.456385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] PATH_INFO: `/v3/projects/27ad835846e74888a8792c08fdebf965/users/24403707b8f54765b1189f511689379e/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.456634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.457287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.457573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.457851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.466902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.468376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1c366-bd3f-463f-b142-d4e2c52190fb service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.470628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27ad835846e74888a8792c08fdebf965, user_id=24403707b8f54765b1189f511689379e, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.473906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.478819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 800/1614] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:34.480430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d850-5b38-4171-acf5-c385e392c9ec None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.480759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d850-5b38-4171-acf5-c385e392c9ec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.480995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153d850-5b38-4171-acf5-c385e392c9ec None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.482326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 801/1615] 10.222.0.46 () {54 vars in 961 bytes} [Wed Jul 6 15:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:34.483448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.483797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.484080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.496165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 815/1616] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/27ad835846e74888a8792c08fdebf965/users/24403707b8f54765b1189f511689379e/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.496883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.512679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.513898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.571412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423f3cd-8f56-4ebb-81d3-917b50cc5a27 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.577624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.578426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.578706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.579119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.579944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.580624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 802/1617] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.581936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.585517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd8716-a6f8-4961-bd14-42af83108ded admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.592140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e62cf-b9e9-4b81-903e-3c1b49cb9787 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.597587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.598486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.598770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.599088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] PATH_INFO: `/v3/projects/e9fb04db8ac9464086cd7f3362e97455/users/68676523b99247bf8f5b9a7553e67f26/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.615095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fb04db8ac9464086cd7f3362e97455, user_id=68676523b99247bf8f5b9a7553e67f26, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.617296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 816/1618] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:34.617813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.621757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9acbf-ade6-4fe7-afc8-df5dc8ebe78c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.622050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9acbf-ade6-4fe7-afc8-df5dc8ebe78c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.622413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9acbf-ade6-4fe7-afc8-df5dc8ebe78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.637220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 803/1619] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/e9fb04db8ac9464086cd7f3362e97455/users/68676523b99247bf8f5b9a7553e67f26/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.637220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.641437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-57c9acbf-ade6-4fe7-afc8-df5dc8ebe78c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.642646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-57c9acbf-ade6-4fe7-afc8-df5dc8ebe78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.700566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:34.714431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 817/1620] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.715877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.716174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.716453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.731280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab6973-5d6e-44bc-8668-0cd85b74a07a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.736471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.737680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.737810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.738295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.738658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] PATH_INFO: `/v3/projects/b1040826386b48b9ac0974c0002cec9f/users/8126a3e25e7e4df29f75b76d8f131e36/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.738961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.741225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1040826386b48b9ac0974c0002cec9f, user_id=8126a3e25e7e4df29f75b76d8f131e36, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.744136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d485e07-bd76-4123-9998-2ac65e3212f2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.757596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 804/1621] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:34 2022] PUT /identity/v3/projects/b1040826386b48b9ac0974c0002cec9f/users/8126a3e25e7e4df29f75b76d8f131e36/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:34.758876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.759178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.759505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.772767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:34.794658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] MFA Rules not processed for user `24403707b8f54765b1189f511689379e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:34.796486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.830553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 818/1622] 10.222.0.46 () {58 vars in 1058 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.832422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.834683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:34.870811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 805/1623] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:34.871579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:34.908561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ffd39-0dad-4b6b-8a30-861c34e10757 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.957452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2be4a-7d7b-478c-a8ed-9e8bb4986a41 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:34.963899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.964792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.965080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.965367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.966075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.968336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc28457-f8a0-4aa6-aefa-71f76cfaeb87 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.981424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:34.982541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.983180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.983667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:34.996446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 806/1624] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:34.996639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:34.997771 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:34.998591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7e17-325a-4b07-b858-efc6606c0fff None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:34.999485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7e17-325a-4b07-b858-efc6606c0fff None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:34.999485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7e17-325a-4b07-b858-efc6606c0fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.010901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 819/1625] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.011587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.026003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afd73e-6a52-4b2b-bac7-7c02cb2c2c98 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.027589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-88cf7e17-325a-4b07-b858-efc6606c0fff None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:35.028852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-88cf7e17-325a-4b07-b858-efc6606c0fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:35.036305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.037089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.037274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.037627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.043488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 807/1626] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:35.043488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.043488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.043488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.060046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.061746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60307c-9f74-4541-8792-792678ff3aca service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.075744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 820/1627] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.077566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478017-c582-4113-9882-d30b11464c8f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.078013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478017-c582-4113-9882-d30b11464c8f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.078475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478017-c582-4113-9882-d30b11464c8f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.084799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 821/1628] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 6 15:24:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:35.085466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] MFA Rules not processed for user `8126a3e25e7e4df29f75b76d8f131e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:35.086109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da478017-c582-4113-9882-d30b11464c8f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.087545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:35.136570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:35.178621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:35.187315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da478017-c582-4113-9882-d30b11464c8f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.194402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 808/1629] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:35.195872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.196683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.197562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.198017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.198524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.209058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8819be-63eb-4cc0-a2cf-7b4207c5abd2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.216893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.218700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.219338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.219794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.220982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.221211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.223361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.224973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6cc29c-a696-469a-9e7d-3912b428caba service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.238506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 809/1630] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:35.239710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.252397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 822/1631] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.259525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d4b0-5b7e-451e-8f94-9f2cf3ac435f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.260889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.261191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.261483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.265043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.267032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.267315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.267776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.286502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.290451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.291633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:35.293425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:35.315922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 823/1632] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:35.316966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.318719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 810/1633] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.319650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.334748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff51a9e-3728-406d-8331-7625ee59aa58 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.341978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.342625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.343430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.343829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] PATH_INFO: `/v3/projects/f48ca307958747cc990287166974794d/users/9896d3207dbf45dd8f61069314b782b1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.368070 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48ca307958747cc990287166974794d, user_id=9896d3207dbf45dd8f61069314b782b1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.372406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f3f0f-bdaf-4a42-8292-8c6678cebbcc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.402514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 811/1634] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:35 2022] PUT /identity/v3/projects/f48ca307958747cc990287166974794d/users/9896d3207dbf45dd8f61069314b782b1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:35.404046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a587-fae7-4bce-9833-49edce97c532 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.407469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a587-fae7-4bce-9833-49edce97c532 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.408123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a587-fae7-4bce-9833-49edce97c532 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.412614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 812/1635] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 6 15:24:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:35.420638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a587-fae7-4bce-9833-49edce97c532 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.421038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a72c34-e273-420c-8b71-4b65a9628dcc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.505125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.505958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.506375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.506661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.541870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a587-fae7-4bce-9833-49edce97c532 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.546598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.547355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.547690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.548120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.642107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.644865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.666463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 824/1636] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.667725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.691396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.692876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb594f-314d-4062-aa5a-dc77c13405b2 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.715255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 813/1637] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.718430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.718728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.719012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.765709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:35.769682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:35.771025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:35.784636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 814/1638] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:35.805754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ec945-b6f4-4d91-9ff1-448403907fa1 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.821026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.821735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.822021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.822299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.824131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.828561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.858207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.863864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 825/1639] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:35.868729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.879310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa8dd5-34b5-4989-a88d-7625c5a3dc55 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.889554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.891451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.891451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.891683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.899247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd71166-9226-49f0-aa51-f0951719ea13 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:35.914282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:35.915205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.920403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.920403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] PATH_INFO: `/v3/projects/f48ca307958747cc990287166974794d/users/9896d3207dbf45dd8f61069314b782b1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.924314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48ca307958747cc990287166974794d, user_id=9896d3207dbf45dd8f61069314b782b1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.926584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:35.927788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.928610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7862bb5-9eb1-460b-b719-7375e833580b service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:35.955911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 826/1640] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:35 2022] PUT /identity/v3/projects/f48ca307958747cc990287166974794d/users/9896d3207dbf45dd8f61069314b782b1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:35.956780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:35.958825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 815/1641] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:35.966577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:35.967700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:35.968118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:35.980107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:36.028793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e20fc-69a5-4221-917c-23750af75544 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:36.029294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] MFA Rules not processed for user `9896d3207dbf45dd8f61069314b782b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:36.033228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:36.034637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:36.035033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:36.035666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:36.036330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:36.130816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:36.132311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:36.154795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 827/1642] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:36.217507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 816/1643] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:36.599287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:36.621668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6ca3-c8c0-47c7-87ff-2229e126bb7a service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:36.637452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:36.637452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:36.637452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:36.637452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:36.668054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:36.669557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:36.691467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 828/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:36.969692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:37.097549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:37.119041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875c919-3609-4467-afc8-c0406f886810 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:37.133404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:37.134674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:37.134963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:37.135264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:37.138549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec28301-8db7-45d7-91da-3be864de34c9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:37.150379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:37.151310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:37.151600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:37.151977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] PATH_INFO: `/v3/users/30c6a87965c349a3aeebeddf65a4c0fa` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:37.154615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c6a87965c349a3aeebeddf65a4c0fa)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:37.155744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:37.155744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jul 06 15:24:37.155744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: Jul 06 15:24:37.156975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:37.268058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:37.270045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e27632-28af-4f05-b8d0-4d56167b4730 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:37.302073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 829/1645] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:37.327574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c6a87965c349a3aeebeddf65a4c0fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:37.328283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c6a87965c349a3aeebeddf65a4c0fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:37.346010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c6a87965c349a3aeebeddf65a4c0fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:37.410906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c6a87965c349a3aeebeddf65a4c0fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:37.456211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c6a87965c349a3aeebeddf65a4c0fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:37.466967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 817/1646] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:36 2022] DELETE /identity/v3/users/30c6a87965c349a3aeebeddf65a4c0fa => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:37.469205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:37.469852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:37.470290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:37.503760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:37.523189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:37.540697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:37.656618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 830/1647] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:37.695970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:37.697964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:37.715608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:37.845601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29ba1-2c34-4b5f-8245-259fba443a10 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:37.852221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:37.852903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:37.853194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:37.853476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:37.873232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:37.874710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347cef9d-4fbb-4ff9-8f4c-53cbdc458e40 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:37.905214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 818/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:39.076018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:39.077547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:39.109562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7771a-c446-4f53-8215-67bf4c0cb8a0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:39.119791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:39.120862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:39.121266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:39.121787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] PATH_INFO: `/v3/projects/8f95961717a844969240b7b718c8e764` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:39.123562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f95961717a844969240b7b718c8e764)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:39.126431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 15:24:39.126431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jul 06 15:24:39.126431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: Jul 06 15:24:39.133350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:39.309595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f95961717a844969240b7b718c8e764', 'request_id': 'req-ccf82720-7b9c-42e1-95f8-b8e995526be7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:39.355552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f95961717a844969240b7b718c8e764', 'request_id': 'req-ccf82720-7b9c-42e1-95f8-b8e995526be7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:39.373500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f95961717a844969240b7b718c8e764', 'request_id': 'req-ccf82720-7b9c-42e1-95f8-b8e995526be7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:39.400421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] The token cache is being invalidate because project 8f95961717a844969240b7b718c8e764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:39.400915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:39.408373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ccf82720-7b9c-42e1-95f8-b8e995526be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:39.410974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 831/1649] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:39 2022] DELETE /identity/v3/projects/8f95961717a844969240b7b718c8e764 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:39.416108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:39.416626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:39.416724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:39.485142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:39.490284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:40.145929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 819/1650] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 725 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:40.148016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.149449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.149864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.191864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:40.201663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:40.217033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 832/1651] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:40.222061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.239593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a8b64-c9ed-4a53-9b14-a50cba407941 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.266935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.275069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.275814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.276101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.279550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:40.281022 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.290492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.339577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 820/1652] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:40.341262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.351149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296288-a048-4d05-8b52-d8c75f79621e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.379045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.379945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.380755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.381169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.382018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.384197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.459531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 833/1653] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:40 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:24:40.460170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.473968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b485e-35fc-41cc-b54c-3d5a7960f76e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.652388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.654896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.655290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.658250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.659741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.664595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.717704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 821/1654] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:40 2022] POST /identity/v3/users => generated 287 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:24:40.719323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.738916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa88c4b-fc82-4326-a398-170ccbfe4afe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.753394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.754317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.754710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.755095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.755997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.759292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.797196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 834/1655] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:40.799300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.816192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6ed9-36e3-4734-a987-e64946179ba2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.831511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.833399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.833712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.833994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.836665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.838333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26ecfd-a16a-4a6f-9d88-f5af5eaefbd4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.869344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 822/1656] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:40.870598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:40.888938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27d2b2-43c9-4295-8967-c543422ff79c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:40.902223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:40.903429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.905415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.905808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] PATH_INFO: `/v3/projects/62e5aa937ecc428895665dcb025bc94e/users/a16a81695ca64500b81e4ad431f31abc/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.931980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e5aa937ecc428895665dcb025bc94e, user_id=a16a81695ca64500b81e4ad431f31abc, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:40.935402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:40.969984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:40.970279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:40.970618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:40.974599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 835/1657] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:40 2022] PUT /identity/v3/projects/62e5aa937ecc428895665dcb025bc94e/users/a16a81695ca64500b81e4ad431f31abc/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:41.040254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] MFA Rules not processed for user `a16a81695ca64500b81e4ad431f31abc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:41.041497 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:41.069368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:41.234350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 823/1658] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:41.240298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:41.312195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:41.366955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cc245-f4ea-4623-9d68-6e91cdd0f899 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:41.373634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:41.374713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:41.375208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:41.375784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:41.376705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:41.379443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:41.455141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 836/1659] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:41 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:41.455716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:41.469615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad18c99-6172-4d2e-9ce9-af8181945ae6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:41.577526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:41.578533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:41.578822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:41.579099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:41.580109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:41.582327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:41.615192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:41.626323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 824/1660] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:41 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:41.637361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917fa1-a3b7-49b7-8e2e-896a97491312 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:41.644661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:41.645388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:41.645674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:41.645954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:41.646783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:41.650389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:41.680803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 837/1661] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:41.685258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:41.704522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e5e6-e9fb-4bd0-8df5-3782ee00f1fb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:41.732814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:41.733753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:41.734040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:41.735050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] PATH_INFO: `/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8/users/6b3467b11d344f469b6f3fe025f2932c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:41.788515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31558b7ea1f14c868fbcc4bd9b691fa8, user_id=6b3467b11d344f469b6f3fe025f2932c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:41.792322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:41.823295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 825/1662] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:41 2022] PUT /identity/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8/users/6b3467b11d344f469b6f3fe025f2932c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:41.827389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:41.845700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5527b4-a35a-4a6b-8a53-d03a80ba47be admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:42.024373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:42.025081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:42.025372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:42.025658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:42.026411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:42.036680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce3b1d-df5d-418b-874c-8c3f98ca74bf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:42.071454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:42.080427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 838/1663] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:42.084592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c98337-1dc9-430a-bf4e-8418f53d95f1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:42.116086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:42.117020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:42.117310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:42.117590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] PATH_INFO: `/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8/users/6b3467b11d344f469b6f3fe025f2932c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:42.119707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=31558b7ea1f14c868fbcc4bd9b691fa8, user_id=6b3467b11d344f469b6f3fe025f2932c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:42.122381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:42.152616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 826/1664] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:42 2022] PUT /identity/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8/users/6b3467b11d344f469b6f3fe025f2932c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:24:42.157302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:42.158233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:42.158762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:42.220863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] MFA Rules not processed for user `6b3467b11d344f469b6f3fe025f2932c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:42.223166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:42.396403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 839/1665] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:42.408257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:42.412108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:42.579132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c58-161d-45de-82e9-fad7c19a1bce admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:42.583068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:42.583795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:42.584102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:42.584698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:42.605550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:42.607065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ad22c-daab-4146-b971-5d6b8703a6a7 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:42.638230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:42.659648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 827/1666] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:43.001909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:43.003457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:43.091310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:43.257229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee2ef-8169-4217-b550-1a0b90c02ab7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:43.265844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:43.267459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:43.267903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:43.268860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] PATH_INFO: `/v3/users/192a3958684e42328536ebff80e351d4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:43.275067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] RBAC: Authorizing `identity:delete_user(user_id=192a3958684e42328536ebff80e351d4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:43.278862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:43.418741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192a3958684e42328536ebff80e351d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.456369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192a3958684e42328536ebff80e351d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.501136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192a3958684e42328536ebff80e351d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.521456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192a3958684e42328536ebff80e351d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.527790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192a3958684e42328536ebff80e351d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.554301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 840/1667] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:43 2022] DELETE /identity/v3/users/192a3958684e42328536ebff80e351d4 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:43.555951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:43.556483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:43.557132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:43.600604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:43.603003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:43.656700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:43.659712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:43.770765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 828/1668] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:43.816382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b0184-bc69-4791-ac01-6fff9751251a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:43.828603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:43.829343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:43.830138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:43.830283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:43.830630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:43.831404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] PATH_INFO: `/v3/users/a16a81695ca64500b81e4ad431f31abc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:43.840220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16a81695ca64500b81e4ad431f31abc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:43.849938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:43.972671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16a81695ca64500b81e4ad431f31abc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:43.975103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:43.976456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251919e4-f9af-4a71-95fe-ff71f6d8ead3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:43.984664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:43.985480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:43.985772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:43.988199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:44.010767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16a81695ca64500b81e4ad431f31abc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:44.015033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:44.020002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:44.029033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16a81695ca64500b81e4ad431f31abc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:44.041134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16a81695ca64500b81e4ad431f31abc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:44.042044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0806d04-bff2-4dc9-b619-21a1d095a430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16a81695ca64500b81e4ad431f31abc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:44.052105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 841/1669] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:43 2022] DELETE /identity/v3/users/a16a81695ca64500b81e4ad431f31abc => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:44.058048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547b79-1618-42bc-b97e-fd922208ba8d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:44.058479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547b79-1618-42bc-b97e-fd922208ba8d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:44.058997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547b79-1618-42bc-b97e-fd922208ba8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:44.060221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 829/1670] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:24:44.081362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-05547b79-1618-42bc-b97e-fd922208ba8d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:44.082773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-05547b79-1618-42bc-b97e-fd922208ba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:44.237414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 842/1671] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:44.799806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:44.809802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bcbef-3474-417b-9e90-2bb6fa848c6e service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:44.963163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:44.964046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:44.964442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:44.964820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:44.986596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:44.989607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:45.014509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 830/1672] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:24:45.880009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:45.880446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:45.880600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:45.943580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] MFA Rules not processed for user `0f5497ff570f4affbbd1b2f9129c24ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:45.947326 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:46.082613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 843/1673] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:46.096762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:46.098206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:46.227936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:46.285012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f24be-c9d1-4ec4-b5f1-00ff58945023 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:46.296710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:46.297769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:46.298059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:46.298339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:46.309489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:46.311676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:46.329959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 831/1674] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:48.138020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:48.140964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:48.160731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21c2f-20ba-4adf-92c1-f1333b0cdb1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:48.168967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:48.170235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:48.170876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:48.171635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] PATH_INFO: `/v3/projects/3bd8f212edb44e65bdf22e130b50cc01` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:48.174101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd8f212edb44e65bdf22e130b50cc01)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:48.180212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:48.341508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd8f212edb44e65bdf22e130b50cc01', 'request_id': 'req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.344426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:48.360794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd8f212edb44e65bdf22e130b50cc01', 'request_id': 'req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.383425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd8f212edb44e65bdf22e130b50cc01', 'request_id': 'req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.442387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] The token cache is being invalidate because project 3bd8f212edb44e65bdf22e130b50cc01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:48.453165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.455833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.457921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 844/1675] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:48 2022] DELETE /identity/v3/projects/3bd8f212edb44e65bdf22e130b50cc01 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:48.461849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:48.482032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f14580-73a3-423a-8727-f487158aa25b service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:48.496082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:48.501049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:48.666939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4ea6-2ac0-4d42-9d59-9f397e8ed52c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:48.679832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:48.681002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:48.682020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:48.682442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] PATH_INFO: `/v3/users/24403707b8f54765b1189f511689379e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:48.684627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24403707b8f54765b1189f511689379e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:48.688077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:48.698720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:48.700965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:48.703380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:48.703903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] PATH_INFO: `/v3/users/9896d3207dbf45dd8f61069314b782b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:48.709981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9896d3207dbf45dd8f61069314b782b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:48.719399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:48.823368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24403707b8f54765b1189f511689379e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.868604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24403707b8f54765b1189f511689379e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.869221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9896d3207dbf45dd8f61069314b782b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.870161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9896d3207dbf45dd8f61069314b782b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.893372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24403707b8f54765b1189f511689379e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.897792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9896d3207dbf45dd8f61069314b782b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.911244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24403707b8f54765b1189f511689379e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.912405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8e61ccc0-cd1f-44d2-a334-9e6b9bab9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24403707b8f54765b1189f511689379e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.928215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 845/1676] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:48 2022] DELETE /identity/v3/users/24403707b8f54765b1189f511689379e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:48.931421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:48.931421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:48.931421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:48.936580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9896d3207dbf45dd8f61069314b782b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.954134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2c1da402-fa65-4c44-8c9d-d289fbb3e4fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9896d3207dbf45dd8f61069314b782b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:48.979019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 832/1677] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:48 2022] DELETE /identity/v3/users/9896d3207dbf45dd8f61069314b782b1 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:48.980758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:48.981199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:48.981603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.023586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] MFA Rules not processed for user `46896be59b73481287235aac3d630f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:49.026705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:49.028698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.047587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.122954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:49.166333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 833/1678] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:49.169167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:49.180731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:49.182384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.253921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 846/1679] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:49.265868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:49.271826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.336380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce32df-d9a5-4b4f-bc77-a3a761226e45 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:49.362762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:49.363871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:49.364356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:49.365517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] PATH_INFO: `/v3/projects/62e5aa937ecc428895665dcb025bc94e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.375535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e5aa937ecc428895665dcb025bc94e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:49.380335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:49.418532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cb706-203e-4005-8a4a-9c1354331f71 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:49.441665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:49.442351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:49.442644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:49.442928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.460617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:49.462617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b0e48-87f2-49f4-aaa2-08d66e808655 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:49.492281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 834/1680] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:49.495540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:49.495833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:49.496120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.499021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e5aa937ecc428895665dcb025bc94e', 'request_id': 'req-79befc09-508d-4198-8825-6ab0a0ccefe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:49.516413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e5aa937ecc428895665dcb025bc94e', 'request_id': 'req-79befc09-508d-4198-8825-6ab0a0ccefe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:49.537107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] MFA Rules not processed for user `46896be59b73481287235aac3d630f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:49.540673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e5aa937ecc428895665dcb025bc94e', 'request_id': 'req-79befc09-508d-4198-8825-6ab0a0ccefe0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:49.546912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.584490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] The token cache is being invalidate because project 62e5aa937ecc428895665dcb025bc94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:49.585637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:49.589469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:49.598319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 847/1681] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:49 2022] DELETE /identity/v3/projects/62e5aa937ecc428895665dcb025bc94e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:24:49.607056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:49.623921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.722493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 835/1682] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:49.737424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:49.740904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.917648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7d455-8b53-4351-b772-bf79addc4d6a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:49.942646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:49.943346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:49.943784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:49.944074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.947751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79befc09-508d-4198-8825-6ab0a0ccefe0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:49.953132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:49.969875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:49.972981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:49.975548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:49.975866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] PATH_INFO: `/v3/users/6b3467b11d344f469b6f3fe025f2932c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:49.981644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3467b11d344f469b6f3fe025f2932c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:49.988563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:49.997051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:49.998678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:50.031994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 836/1683] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:50.127962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3467b11d344f469b6f3fe025f2932c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.162933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3467b11d344f469b6f3fe025f2932c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.190287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3467b11d344f469b6f3fe025f2932c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.193102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:50.201937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3467b11d344f469b6f3fe025f2932c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.203488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3467b11d344f469b6f3fe025f2932c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.234462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 848/1684] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:49 2022] DELETE /identity/v3/users/6b3467b11d344f469b6f3fe025f2932c => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:50.341270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:50.342779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:50.362535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:50.364041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:50.381958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3281f17-659e-4b46-85c5-e2da140582f9 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:50.464370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:50.536281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63837c6c-b388-4cf0-b5fa-f22eef69779b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:50.554388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:50.555333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.555623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.555910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] PATH_INFO: `/v3/projects/27ad835846e74888a8792c08fdebf965` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.558293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27ad835846e74888a8792c08fdebf965)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:50.560294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:50.613031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:50.616287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.616588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.617140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] PATH_INFO: `/v3/projects/f48ca307958747cc990287166974794d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.620969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48ca307958747cc990287166974794d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:50.630552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:50.682060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27ad835846e74888a8792c08fdebf965', 'request_id': 'req-b706d47a-28f2-4f26-b449-9b416ec5f461'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.702916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27ad835846e74888a8792c08fdebf965', 'request_id': 'req-b706d47a-28f2-4f26-b449-9b416ec5f461'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.716983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27ad835846e74888a8792c08fdebf965', 'request_id': 'req-b706d47a-28f2-4f26-b449-9b416ec5f461'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.741957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] The token cache is being invalidate because project 27ad835846e74888a8792c08fdebf965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:50.742415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.743288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b706d47a-28f2-4f26-b449-9b416ec5f461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.745093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 849/1685] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:50 2022] DELETE /identity/v3/projects/27ad835846e74888a8792c08fdebf965 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:24:50.758137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128108ed-9c8a-4e0f-bb54-ea0bceb8cbd4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.758570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128108ed-9c8a-4e0f-bb54-ea0bceb8cbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.758972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128108ed-9c8a-4e0f-bb54-ea0bceb8cbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.784798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48ca307958747cc990287166974794d', 'request_id': 'req-09d07d18-1018-4d05-b7b3-2b50d43fce6a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.802361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48ca307958747cc990287166974794d', 'request_id': 'req-09d07d18-1018-4d05-b7b3-2b50d43fce6a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.802718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-128108ed-9c8a-4e0f-bb54-ea0bceb8cbd4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:50.806300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-128108ed-9c8a-4e0f-bb54-ea0bceb8cbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:50.823102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48ca307958747cc990287166974794d', 'request_id': 'req-09d07d18-1018-4d05-b7b3-2b50d43fce6a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.837364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] The token cache is being invalidate because project f48ca307958747cc990287166974794d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:50.837929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.838766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09d07d18-1018-4d05-b7b3-2b50d43fce6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:50.850353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 837/1686] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:50 2022] DELETE /identity/v3/projects/f48ca307958747cc990287166974794d => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:50.852280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.852803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.853232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.879280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:50.881006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:50.914454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 850/1687] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:50.916511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ad3-3d85-4828-b41c-9ba51176940f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.916917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ad3-3d85-4828-b41c-9ba51176940f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.917342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a5ad3-3d85-4828-b41c-9ba51176940f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.919704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 851/1688] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 6 15:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:50.926795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0802e8-f0b8-4afe-9225-ce1f47440512 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.927708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0802e8-f0b8-4afe-9225-ce1f47440512 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.927824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0802e8-f0b8-4afe-9225-ce1f47440512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.963662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cc0802e8-f0b8-4afe-9225-ce1f47440512 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:50.965361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc0802e8-f0b8-4afe-9225-ce1f47440512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:50.976776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 852/1689] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:50.978479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:50.979113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:50.979598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:50.984117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 838/1690] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:51.003296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.019850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:51.026950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2126c-bc8d-4d8d-93a3-b24e559c7b2b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.043729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.044974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.045395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.046985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.047792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:51.047906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:51.048738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.052837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b89143-e3aa-423a-9853-735bc209b787 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.053685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:51.084814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 839/1691] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:51.090122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.090313 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.090651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.124677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 853/1692] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:51.133557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.137045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:51.141804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:51.149954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6dcc1-b2e0-4376-ab02-d5f0e3ed9be6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.156725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.158108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.158661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.159294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.160177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.165850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.166851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 840/1693] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:51.167949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.179683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9ecb8-be22-4401-8c24-f0e28c9cd9c4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.185552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.186705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.187033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.187288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.188065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:51.188399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.190649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.204589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 841/1694] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:51.209765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.211759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:51.212689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 854/1695] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:51 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:51.213794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.242521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e1c29-4977-4dc5-a143-1e651ff3f995 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.303280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:51.359157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cebbd0-98c7-4e34-bfc3-f1066aeff361 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.371615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.372781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.373456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.376188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.376595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.380917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.389698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.390440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.390917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.391604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.420724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.424375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.443167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 842/1696] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:51.443167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.454977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 855/1697] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:51.460805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.482011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a0e5e-8b2e-4988-85bc-cae92f7216b6 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.619532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995926c8-ae5e-4faa-8e30-f8c4744348ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.627756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.628516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.628809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.629088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.631200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.631529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.633309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.633978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.634312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.635898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.636862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.639411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.691623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 856/1698] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:51 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:51.695345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 843/1699] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:51 2022] POST /identity/v3/users => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:51.697736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.698048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.703820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:51.728999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bb8a5-235b-4fc5-a05e-eab89be1d519 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.731519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990d71b-2f51-40b4-b36a-bac6186465a3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.735276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.736913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.747181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.747910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.748201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.748499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.749376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.751004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.753334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.768204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.770887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.775750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.785876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 844/1700] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:51.789315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.814617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 857/1701] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:51.817651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.819465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de0fa1-91f8-4fcd-8474-b6c03c7f9daf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.835351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.837936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.838059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.838350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.841741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067fce-be44-4599-a984-4030e2226005 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.848048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.860161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.861165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.862630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.864420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.865301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.869712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.871508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.901861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 858/1702] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:51.903309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.903721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 845/1703] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:51.906408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:51.934721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeb46b-dd72-486f-b24d-e54030385dea admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.935600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9ff0e-c275-4b69-98bb-4f9c7a5159be admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:51.946831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.947788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.948074 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.948459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] PATH_INFO: `/v3/projects/353433f606a645378d87e814e8de284e/users/c0a231e263364055b5f01e2445e30912/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.953934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:51.955104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:51.955711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:51.956152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] PATH_INFO: `/v3/projects/f555cf9f335a4d48a15d2709b857a6d1/users/c7702ef2d8c240a1b10bce8590ef495e/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:51.967919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] RBAC: Authorizing `identity:create_grant(project_id=353433f606a645378d87e814e8de284e, user_id=c0a231e263364055b5f01e2445e30912, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:51.972345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e7eab-179f-4d7b-b11a-981b442b379d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:51.996910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f555cf9f335a4d48a15d2709b857a6d1, user_id=c7702ef2d8c240a1b10bce8590ef495e, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:52.002020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:52.010151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 846/1704] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:51 2022] PUT /identity/v3/projects/353433f606a645378d87e814e8de284e/users/c0a231e263364055b5f01e2445e30912/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:52.026410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c311-b731-4d38-89fd-e5d9ae4c249c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.026739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c311-b731-4d38-89fd-e5d9ae4c249c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.026973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c311-b731-4d38-89fd-e5d9ae4c249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.045132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 859/1705] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:51 2022] PUT /identity/v3/projects/f555cf9f335a4d48a15d2709b857a6d1/users/c7702ef2d8c240a1b10bce8590ef495e/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:52.045453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:52.050363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.077004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3644c311-b731-4d38-89fd-e5d9ae4c249c None None] MFA Rules not processed for user `c0a231e263364055b5f01e2445e30912`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:52.078265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3644c311-b731-4d38-89fd-e5d9ae4c249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.223056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327c69-4e7e-49a6-84d6-9c83f89ec6cf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:52.230724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:52.234262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:52.270080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.271063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.272050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] PATH_INFO: `/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.277532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31558b7ea1f14c868fbcc4bd9b691fa8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:52.279181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 847/1706] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:52.280599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:52.285496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.286007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.286172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.326595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:52.331078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.360630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 848/1707] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:52.362814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:52.366254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.385359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31558b7ea1f14c868fbcc4bd9b691fa8', 'request_id': 'req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:52.402641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31558b7ea1f14c868fbcc4bd9b691fa8', 'request_id': 'req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:52.420263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31558b7ea1f14c868fbcc4bd9b691fa8', 'request_id': 'req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:52.455951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] The token cache is being invalidate because project 31558b7ea1f14c868fbcc4bd9b691fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:52.457280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:52.458543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1bba5eac-3fdd-4121-95c5-2a4be9b236c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:52.461136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 860/1708] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:52 2022] DELETE /identity/v3/projects/31558b7ea1f14c868fbcc4bd9b691fa8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:52.466558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49767-59a1-4858-88ad-872feae10219 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.466558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49767-59a1-4858-88ad-872feae10219 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.468626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49767-59a1-4858-88ad-872feae10219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.497747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936220a-8856-4edc-8c6d-aea561514883 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:52.540952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-14d49767-59a1-4858-88ad-872feae10219 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:52.542229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14d49767-59a1-4858-88ad-872feae10219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.609976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:52.610667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.610960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.611269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.617221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.707749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:52.766317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 861/1709] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:24:52.767110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49767-59a1-4858-88ad-872feae10219 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:52.772824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14d49767-59a1-4858-88ad-872feae10219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.812262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:52.814085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28376f9-a429-4eaa-b90b-92c87d23e1e4 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:52.844355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 849/1710] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:52.848869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.849474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.856524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.881642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:52.883041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.919867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 850/1711] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:52.921407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:52.925271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:52.953170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4bca7-9e3c-4679-9c06-692dada6ff49 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:52.965963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49767-59a1-4858-88ad-872feae10219 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:52.975713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:52.979150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.981194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.982259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:52.983137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:52.983558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:52.983959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.985844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] PATH_INFO: `/v3/users/c0a231e263364055b5f01e2445e30912` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:52.992314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.005031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a231e263364055b5f01e2445e30912)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.006995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.051685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.056138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.090801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 862/1712] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:53.091907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.123291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5594d01-f3d8-4197-b6c4-a6db9e9d0357 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:53.130107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:53.131079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.131414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.131700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.132604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:53.132925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.134818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.138957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a231e263364055b5f01e2445e30912'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:53.145261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a231e263364055b5f01e2445e30912'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:53.150226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:53.156689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 863/1713] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:53.158399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.161698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.170386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a231e263364055b5f01e2445e30912'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:53.195612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a231e263364055b5f01e2445e30912'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:53.244692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0f6602fa-8ead-485f-bbea-cf054eed9fbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a231e263364055b5f01e2445e30912'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:53.258446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff450e31-dbeb-43e6-961e-d737011399c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:53.258902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 851/1714] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:24:52 2022] DELETE /identity/v3/users/c0a231e263364055b5f01e2445e30912 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:53.260031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526450-08db-4525-90a5-a5f3cd30e4f2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.261656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526450-08db-4525-90a5-a5f3cd30e4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.262636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526450-08db-4525-90a5-a5f3cd30e4f2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.269560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 852/1715] 10.222.0.46 () {54 vars in 960 bytes} [Wed Jul 6 15:24:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:53.270661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a05f-acba-4d03-8f53-56ec8245362a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.271029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a05f-acba-4d03-8f53-56ec8245362a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.271419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a05f-acba-4d03-8f53-56ec8245362a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.325306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3451a05f-acba-4d03-8f53-56ec8245362a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:53.326742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3451a05f-acba-4d03-8f53-56ec8245362a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.467443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:53.468143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.468435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.468723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.471669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.481959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 853/1716] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:53.483457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24a4-ac24-4063-9cc3-56dccd639863 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.484368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.495534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24a4-ac24-4063-9cc3-56dccd639863 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.497293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24a4-ac24-4063-9cc3-56dccd639863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.535668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 864/1717] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:53 2022] POST /identity/v3/projects => generated 367 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:53.538240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-afef24a4-ac24-4063-9cc3-56dccd639863 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:53.540168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.545098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-afef24a4-ac24-4063-9cc3-56dccd639863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.675181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7cae2-79d2-48c6-aec2-8094db6a19ea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:53.688002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:53.689063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.689063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.691994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.693254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.697259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.715439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:53.756483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 865/1718] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:53 2022] POST /identity/v3/users => generated 284 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:53.756483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.757946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.783377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 854/1719] 10.222.0.46 () {58 vars in 1057 bytes} [Wed Jul 6 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:53.790455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.790879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.791180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.829387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] MFA Rules not processed for user `46896be59b73481287235aac3d630f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:53.832977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:53.870514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daddd8d-1a2d-4ddd-a0ac-870e38c99409 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:53.881217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:53.881922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.882229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.882516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.905021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.909955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:53.945870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 866/1720] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:53.948458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.978766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bce2e-51eb-424b-8525-0dcb87a91419 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:53.989048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 855/1721] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:53.990085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:53.990314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:53.991070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:53.991393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:53.991680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:53.994893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:53.996593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.018733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda2206-79e9-4895-b2ad-5a2745ca1d05 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.026594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 867/1722] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:54.027692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.028384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.028751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.028965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.029756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:54.034226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.069917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6121e69-088e-4481-bf11-054e10162a74 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.086017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:54.090410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.093999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214e484-ff94-4f0b-acce-a85173adbbdf service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.100674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.101386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.101673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.102036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.102826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.104577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52980682-f04b-47ce-abe8-96dceb6d4783 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.115652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 856/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:54.118871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.119280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.119651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.155154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 868/1724] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:54.161382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab6108-efab-4ca5-8d33-c56895e0f04c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.161690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab6108-efab-4ca5-8d33-c56895e0f04c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.161981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab6108-efab-4ca5-8d33-c56895e0f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.182420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] MFA Rules not processed for user `68676523b99247bf8f5b9a7553e67f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:54.184278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.260458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4eab6108-efab-4ca5-8d33-c56895e0f04c None None] MFA Rules not processed for user `213ac803c2cb4cbfb4d73ec4fb07d552`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:54.262159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4eab6108-efab-4ca5-8d33-c56895e0f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.332154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 857/1725] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:54.332154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:54.349007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788062-1b0a-4aad-87bf-1f317b644d29 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.355426 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.356359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.356650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.356991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] PATH_INFO: `/v3/projects/fb72104aa6514e5d81204f3af616a93b/users/8843efdb7f144992988d9789ce955b22/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.379524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb72104aa6514e5d81204f3af616a93b, user_id=8843efdb7f144992988d9789ce955b22, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.382886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.428429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 858/1726] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:54 2022] PUT /identity/v3/projects/fb72104aa6514e5d81204f3af616a93b/users/8843efdb7f144992988d9789ce955b22/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:54.429950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:54.435652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.450235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 869/1727] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:54.451750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ad94-37e0-4b12-8876-1a66ed3c5553 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.452049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ad94-37e0-4b12-8876-1a66ed3c5553 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.452547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0ad94-37e0-4b12-8876-1a66ed3c5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.494167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-77f0ad94-37e0-4b12-8876-1a66ed3c5553 None None] MFA Rules not processed for user `8843efdb7f144992988d9789ce955b22`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:54.495680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-77f0ad94-37e0-4b12-8876-1a66ed3c5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.557641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee085a07-cb6c-4cd8-ad7a-b5ace26ef3dc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.561732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:54.566629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.568300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.568750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.569275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.587743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 870/1728] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:54.589640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.589966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.590301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.636021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] MFA Rules not processed for user `213ac803c2cb4cbfb4d73ec4fb07d552`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:54.650140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:54.662438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.664858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.672334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:54.678610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 859/1729] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:54.679728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:54.788940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 871/1730] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:54.878719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:54.890706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cfb22-95c4-43d2-98ec-18f263c3fb60 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.908836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9b4d7-3320-4c59-8cbe-9c928e9218ca None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:54.918440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.919134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.919465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.919744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.920451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.927100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:54.931569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:54.935800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:54.936393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:54.937956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:54.970577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:54.976858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613e8be-2ac7-4a6f-adf0-e8dbef5e1abd service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:55.015487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 860/1731] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:54 2022] POST /identity/v3/projects => generated 367 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:55.019344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:55.041870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 872/1732] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:55.105181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:55.105693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:55.106289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:55.159098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] MFA Rules not processed for user `0f5497ff570f4affbbd1b2f9129c24ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:55.166231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:55.292566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524e568-5100-494c-91dc-be9c2a6a2bdc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:55.313941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:55.320356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:55.320607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:55.320872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:55.321609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:55.331040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:55.380678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 873/1733] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:55.404868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 861/1734] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:55 2022] POST /identity/v3/users => generated 290 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:24:55.407801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:55.452870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f873-7c33-43dd-a224-e1e795501cde None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:55.487721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:55.488509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:55.489084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:55.489414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:55.490265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:55.503177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:55.548283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 874/1735] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:55.553231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:55.587249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1ed65-78f2-4a98-bd91-21d74353fc49 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:55.606050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:55.606983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:55.607298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:55.612302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] PATH_INFO: `/v3/projects/3238b75dc54f4895ae2250cad222fe82/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:55.690623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3238b75dc54f4895ae2250cad222fe82, user_id=b4a0fb9c77aa41ba8cd777fd3e2fcc21, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:55.710184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:55.752420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 862/1736] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:55 2022] PUT /identity/v3/projects/3238b75dc54f4895ae2250cad222fe82/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:55.756311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:55.771580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e3f21-c6bf-4a54-b45f-336b2b7e2684 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:55.990747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:55.995038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:55.995655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:55.997807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.000648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:56.005383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98492eb-5f57-4074-a589-a25d7493cdfa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:56.043434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 875/1737] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:56.047719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:56.068758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b21db7-9281-458d-b29b-5dabcc8ac398 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:56.077798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:56.086432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:56.092124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:56.092481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] PATH_INFO: `/v3/projects/3238b75dc54f4895ae2250cad222fe82/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.094738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3238b75dc54f4895ae2250cad222fe82, user_id=b4a0fb9c77aa41ba8cd777fd3e2fcc21, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:56.098126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:56.126692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 863/1738] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:56 2022] PUT /identity/v3/projects/3238b75dc54f4895ae2250cad222fe82/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:56.137789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77f8b-a746-4ec4-8bb3-cfbf7691c253 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:56.138434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77f8b-a746-4ec4-8bb3-cfbf7691c253 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:56.138967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f77f8b-a746-4ec4-8bb3-cfbf7691c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.205304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:56.208762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c5f77f8b-a746-4ec4-8bb3-cfbf7691c253 None None] MFA Rules not processed for user `b4a0fb9c77aa41ba8cd777fd3e2fcc21`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:56.210194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c5f77f8b-a746-4ec4-8bb3-cfbf7691c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:56.324570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 876/1739] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:56.338009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:56.339772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:56.425319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:56.459346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ed59a-c4aa-4c19-b55b-c7b2d8351632 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:56.464685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:56.466026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:56.466284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:56.466672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.500879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:56.505648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:56.541688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 864/1740] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:56.662268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa9af-de43-4b1e-bd82-13fa8b0d6e80 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:56.662763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa9af-de43-4b1e-bd82-13fa8b0d6e80 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:56.662929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efa9af-de43-4b1e-bd82-13fa8b0d6e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.737938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-03efa9af-de43-4b1e-bd82-13fa8b0d6e80 None None] MFA Rules not processed for user `c7702ef2d8c240a1b10bce8590ef495e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:56.739410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-03efa9af-de43-4b1e-bd82-13fa8b0d6e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:56.933632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 877/1741] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:56.937203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:56.938932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:56.946577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:56.946867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:56.950871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:56.959432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 878/1742] 10.222.0.46 () {54 vars in 912 bytes} [Wed Jul 6 15:24:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:24:56.960494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:56.968690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:57.262834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701c118-e1a8-45aa-a974-65325c1c90be service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:57.291901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:57.292601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.292891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.293172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.301004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:57.308796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ea151-76d2-4002-af07-79aa2da786fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:57.326322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8d56c-aa1a-473c-86c0-47800915bbac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:57.356570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:57.357502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.357891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.358274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] PATH_INFO: `/v3/projects/353433f606a645378d87e814e8de284e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.359914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=353433f606a645378d87e814e8de284e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:57.364659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:57.413015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 865/1743] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:56 2022] POST /identity/v3/projects => generated 405 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:57.421509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7902b-4d89-4fa9-ad01-ce690d043a35 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.421969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7902b-4d89-4fa9-ad01-ce690d043a35 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.422496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7902b-4d89-4fa9-ad01-ce690d043a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.495743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-76e7902b-4d89-4fa9-ad01-ce690d043a35 None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:57.502427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-76e7902b-4d89-4fa9-ad01-ce690d043a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:57.505665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:57.558468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353433f606a645378d87e814e8de284e', 'request_id': 'req-770bcf19-995a-4140-814a-d8e40c13b081'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:57.584606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353433f606a645378d87e814e8de284e', 'request_id': 'req-770bcf19-995a-4140-814a-d8e40c13b081'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:57.601956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353433f606a645378d87e814e8de284e', 'request_id': 'req-770bcf19-995a-4140-814a-d8e40c13b081'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:57.641025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] The token cache is being invalidate because project 353433f606a645378d87e814e8de284e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:24:57.641735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:57.645234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:24:57.656213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 879/1744] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:24:56 2022] DELETE /identity/v3/projects/353433f606a645378d87e814e8de284e => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:57.659087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:57.661907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:57.700861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 866/1745] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:57.700861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.700861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.700861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.755372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:57.758198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:57.843529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bcf19-995a-4140-814a-d8e40c13b081 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:57.854682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:57.855654 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.856297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.857536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.857691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:57.861141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f294998-b858-4680-b2ff-5f1d740c23e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:57.932175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 880/1746] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:57 2022] POST /identity/v3/users => generated 309 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:57.938555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0fdf-f0ba-4a6f-aa35-dc0fd642e46b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:57.938750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0fdf-f0ba-4a6f-aa35-dc0fd642e46b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:57.939853 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0fdf-f0ba-4a6f-aa35-dc0fd642e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:57.960464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 867/1747] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:57.976040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:57.984726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:57.989395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-602e0fdf-f0ba-4a6f-aa35-dc0fd642e46b None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:57.990790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-602e0fdf-f0ba-4a6f-aa35-dc0fd642e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:58.034958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff472b-84d8-4c14-93d2-fcacb17306b2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.051814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.052682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.053084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.053437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.072810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.074546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.097838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:58.112630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 868/1748] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:24:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:24:58.123286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.168189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5eeb5-26dd-4536-819d-7d76c88a4a23 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.195644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.196379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.198208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.198208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.198405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.200043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.221547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 881/1749] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:58.221547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.221547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.221547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.239634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 869/1750] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:58.244342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.273837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ade603-299c-4aeb-ac8f-dc47d6e702e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.291185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.292148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.292440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.292730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] PATH_INFO: `/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9/users/27251177bf404a258bdb5464508534ce/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.294768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:58.298584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:58.330740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 882/1751] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:24:58.330740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.363942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59064f6-1529-431d-bd5b-89f792db6f18 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.372659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.373404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.373625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.373903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.374658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:24:58.374968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.376218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8747dbebeebe4b599d4d08ce9c40c4d9, user_id=27251177bf404a258bdb5464508534ce, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.381196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.384898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.420726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 883/1752] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:24:58.421792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.450115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 870/1753] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:58 2022] PUT /identity/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9/users/27251177bf404a258bdb5464508534ce/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:58.453423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac5c8d-22e7-4528-8a22-e0eb6ed3b00b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.454432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.526825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:58.619013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.621507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.621802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.623534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.624370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.636096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.697152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 884/1754] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:58.698314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.700593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:58.702458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523ace4-e2c0-4e93-b9eb-e6a4d9c7db09 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.850257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.850951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.851258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.851547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.852296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.853915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.881259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 871/1755] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:58.881259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:58.900314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc28d18-8708-4e54-81ed-ed18f2477b98 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.915070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2851-d06d-4acd-a336-f8af1387422b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:58.916891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.917609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.917898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.918182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.918891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:58.920874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:58.933767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:58.936162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:58.939544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:58.945357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:58.967063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 872/1756] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:24:58 2022] POST /identity/v3/users => generated 293 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:24:58.967913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:59.010149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ae601-feef-4100-9266-4d4ec9ac1fd2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:59.026957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:59.028198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:59.028637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:59.029075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] PATH_INFO: `/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9/users/27251177bf404a258bdb5464508534ce/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:59.033753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8747dbebeebe4b599d4d08ce9c40c4d9, user_id=27251177bf404a258bdb5464508534ce, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:59.047380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:59.069749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 873/1757] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:58 2022] PUT /identity/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9/users/27251177bf404a258bdb5464508534ce/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:59.070890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:59.182221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:24:59.185854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:59.191944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:59.228318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 885/1758] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:24:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:24:59.229458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796aa00-b10b-438d-95cd-215c27af872c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:59.259777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:59.261005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:59.261299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:59.261581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:59.263888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:59.265547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:59.325109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 874/1759] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:59.331437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:59.359341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42003c5-5ec4-4bdb-b892-8d8fd02559d0 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:59.389586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:59.390471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:59.391625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:59.394485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:59.395462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:59.401891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d84bc2-9d48-4eb4-a885-4d30cb9c5d0d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:59.438397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 886/1760] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:24:59.444733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:24:59.461782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d821-0deb-4b3d-9b83-8a7013df1756 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:24:59.490578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:24:59.494293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:59.494791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:59.496059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] PATH_INFO: `/v3/projects/b0d1cb88d7d247f58669117081033ea9/users/3c94a216fd174ff08dbd798d986f1f31/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:59.573029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d1cb88d7d247f58669117081033ea9, user_id=3c94a216fd174ff08dbd798d986f1f31, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:24:59.594390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:24:59.649835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 875/1761] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:24:59 2022] PUT /identity/v3/projects/b0d1cb88d7d247f58669117081033ea9/users/3c94a216fd174ff08dbd798d986f1f31/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:24:59.658677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f480c-13db-4e07-a2d0-90c18846344e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:24:59.659011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f480c-13db-4e07-a2d0-90c18846344e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:24:59.659327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f480c-13db-4e07-a2d0-90c18846344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:24:59.730188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cc4f480c-13db-4e07-a2d0-90c18846344e None None] MFA Rules not processed for user `3c94a216fd174ff08dbd798d986f1f31`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:24:59.737130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc4f480c-13db-4e07-a2d0-90c18846344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:24:59.883885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:00.018398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 887/1762] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:00.023956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:00.026654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:00.245082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:00.248093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:00.248640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:00.254256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcddd4-a9bb-46fa-80f9-dd30cac349f6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:00.266358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:00.267500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:00.268039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:00.268605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:00.300791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:00.302750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:00.340842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:00.342113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:00.344872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 876/1763] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:00.528559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:00.530589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:00.616995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 888/1764] 10.222.0.16 () {58 vars in 1015 bytes} [Wed Jul 6 15:25:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:00.816274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:00.835171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78094304-ad04-4eb5-8056-76cefcb015c8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:00.850310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:00.853637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:00.873472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:00.874803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:00.898517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa86150-e3e5-47a3-9b46-4b5e8fb9c3a7 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:00.927111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:00.940188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97bace-acc1-4507-8bed-73f29f21bfba service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:00.945958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:00.949585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:00.949891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:00.956249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] PATH_INFO: `/v3/users/8843efdb7f144992988d9789ce955b22` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:00.960101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] RBAC: Authorizing `identity:delete_user(user_id=8843efdb7f144992988d9789ce955b22)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:00.974979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:00.987000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 889/1765] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:01.233465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8843efdb7f144992988d9789ce955b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:01.262430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8843efdb7f144992988d9789ce955b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:01.308199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8843efdb7f144992988d9789ce955b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:01.316145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:01.367503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8843efdb7f144992988d9789ce955b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:01.399297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8843efdb7f144992988d9789ce955b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:01.424163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0132ed-460c-4fd6-9253-4ad98609ce22 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:01.424742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0132ed-460c-4fd6-9253-4ad98609ce22 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:01.424987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0132ed-460c-4fd6-9253-4ad98609ce22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:01.432880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 877/1766] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:00 2022] DELETE /identity/v3/users/8843efdb7f144992988d9789ce955b22 => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:01.465049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ca0132ed-460c-4fd6-9253-4ad98609ce22 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:01.482718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ca0132ed-460c-4fd6-9253-4ad98609ce22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:01.659136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 890/1767] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:01.721052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:01.724624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:01.938335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee04a-9502-4899-b735-fdcc3efef4db admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:01.948054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:01.949106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:01.949619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:01.950207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:01.992326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:01.995885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd570ce-18b3-4a58-b509-7bdf214d2619 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:02.021363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 878/1768] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:02.686741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:02.688022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:02.688022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:02.729194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] MFA Rules not processed for user `68676523b99247bf8f5b9a7553e67f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:02.741496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:02.926476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 891/1769] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:03.396419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:03.397015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:03.397015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:03.457261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] MFA Rules not processed for user `27251177bf404a258bdb5464508534ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:03.459605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:03.636042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 879/1770] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:03.916892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:03.919493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:03.951413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b7a44-d655-4b25-afd3-a40d9bb0ae23 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:03.993820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:03.996396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:03.996940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:03.999592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] PATH_INFO: `/v3/projects/fb72104aa6514e5d81204f3af616a93b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:04.003710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb72104aa6514e5d81204f3af616a93b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:04.010826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:04.074364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:04.157508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb72104aa6514e5d81204f3af616a93b', 'request_id': 'req-2f038357-1f50-4f51-8d41-27ec0cfddfed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.177473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb72104aa6514e5d81204f3af616a93b', 'request_id': 'req-2f038357-1f50-4f51-8d41-27ec0cfddfed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.206721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb72104aa6514e5d81204f3af616a93b', 'request_id': 'req-2f038357-1f50-4f51-8d41-27ec0cfddfed'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.234029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] The token cache is being invalidate because project fb72104aa6514e5d81204f3af616a93b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:04.234648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.235468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f038357-1f50-4f51-8d41-27ec0cfddfed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.243472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 892/1771] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:03 2022] DELETE /identity/v3/projects/fb72104aa6514e5d81204f3af616a93b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:04.244509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:04.261521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50702136-ed8b-4b6e-88e0-41875e533527 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:04.369553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:04.370615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:04.370970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:04.374485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] PATH_INFO: `/v3/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:04.378166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a0fb9c77aa41ba8cd777fd3e2fcc21)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:04.382107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:04.388787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:04.478649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a0fb9c77aa41ba8cd777fd3e2fcc21'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.484496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abc4d3-7022-4d7e-8871-ebf6931371e0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:04.484763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a0fb9c77aa41ba8cd777fd3e2fcc21'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.485206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abc4d3-7022-4d7e-8871-ebf6931371e0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:04.485497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abc4d3-7022-4d7e-8871-ebf6931371e0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:04.487758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 893/1772] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:25:04.496728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:04.497016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:04.497490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:04.503774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a0fb9c77aa41ba8cd777fd3e2fcc21'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.521757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a0fb9c77aa41ba8cd777fd3e2fcc21'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.530604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a0fb9c77aa41ba8cd777fd3e2fcc21'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:04.539911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 880/1773] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:04 2022] DELETE /identity/v3/users/b4a0fb9c77aa41ba8cd777fd3e2fcc21 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:04.552471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:04.554174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:04.811895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 894/1774] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:05.198842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:05.201173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:05.245758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decde10d-e315-487a-af58-656f887cbacc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:05.261189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:05.262255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.262784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.263770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.276967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:05.278627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:05.298255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:05.300762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec52e3-2395-40e1-8601-f5b66063bb3f service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:05.320181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 881/1775] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:05.424736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fabc1-bac4-4e52-84d0-b55efadc68e1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:05.435552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:05.436634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.437378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.438176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] PATH_INFO: `/v3/projects/3238b75dc54f4895ae2250cad222fe82` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.439962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3238b75dc54f4895ae2250cad222fe82)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:05.442967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:05.498797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:05.509837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3238b75dc54f4895ae2250cad222fe82', 'request_id': 'req-bf38ec23-23f9-41b9-89a0-a06222463d2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:05.518968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3238b75dc54f4895ae2250cad222fe82', 'request_id': 'req-bf38ec23-23f9-41b9-89a0-a06222463d2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:05.534121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3238b75dc54f4895ae2250cad222fe82', 'request_id': 'req-bf38ec23-23f9-41b9-89a0-a06222463d2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:05.558296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] The token cache is being invalidate because project 3238b75dc54f4895ae2250cad222fe82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:05.559646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:05.561237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf38ec23-23f9-41b9-89a0-a06222463d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:05.568757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.569305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.569398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.573705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 895/1776] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:05 2022] DELETE /identity/v3/projects/3238b75dc54f4895ae2250cad222fe82 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:05.596510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:05.598224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:05.712196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.712526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.712847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.723849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 882/1777] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:05.750826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:05.752318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:05.785512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:05.796705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 896/1778] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:05.797929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b82c-ae22-4dc5-9812-f41c05347102 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:05.813440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:05.813813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.814144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.814429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.815181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:05.815529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:05.818553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5738749-df17-4a4b-994d-aa61c2766812 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:05.877725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 883/1779] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:05.882153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:05.906449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304a5d2-21fe-46d1-83da-ddf2954a4c6a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:05.938509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575d9f-3bdb-4744-8371-9940f4b402c3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.939002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575d9f-3bdb-4744-8371-9940f4b402c3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.939805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29575d9f-3bdb-4744-8371-9940f4b402c3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.942332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 884/1780] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:25:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:25:05.951499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:05.952279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:05.952754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:05.953040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:05.953767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:05.958610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.029728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.030057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.030841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.062786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 897/1781] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:05 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:06.066467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:06.115907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:06.117205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:06.367373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:06.434296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 885/1782] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:25:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:06.469248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f255293-63ba-4e36-82ec-d2d2dd6a57fa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:06.497717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:06.502672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:06.507281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:06.508607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.509945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.510716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.511827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:06.521442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.557383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4726dd1-1162-4185-b086-b7fdf74e9b64 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:06.566323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 898/1783] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:06 2022] POST /identity/v3/users => generated 284 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:06.571913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:06.572278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:06.572937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.573550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.574008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.593042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:06.594726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.609699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8aced1-a354-4c4d-94b7-f14fe95a93c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:06.620495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 886/1784] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:06.621847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:06.623951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.624449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.625796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.628346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:06.634024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.674117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 899/1785] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:06.675369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:06.707183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5f75-13c7-4b3c-91d6-b65f08d3f93b service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:06.739830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:06.749740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.750142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.750312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.753809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:06.755524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafde6-68a7-48de-bd7f-b24c4182301d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.811416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 887/1786] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:06.814262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:06.837725 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034934-5a6b-4a71-ad73-ce3d701bc007 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:06.849029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:06.850286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.850674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.851100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] PATH_INFO: `/v3/projects/ebc86c40e47c4945976ff6dd7048b49f/users/5a8f09651080477494f5d1bf1d155857/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:06.903145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc86c40e47c4945976ff6dd7048b49f, user_id=5a8f09651080477494f5d1bf1d155857, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:06.905935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:06.951721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 900/1787] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:06 2022] PUT /identity/v3/projects/ebc86c40e47c4945976ff6dd7048b49f/users/5a8f09651080477494f5d1bf1d155857/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:06.952843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c957ec-3073-434f-bd19-32cf301cbf8e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:06.953141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c957ec-3073-434f-bd19-32cf301cbf8e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:06.953418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c957ec-3073-434f-bd19-32cf301cbf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.027744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-66c957ec-3073-434f-bd19-32cf301cbf8e None None] MFA Rules not processed for user `5a8f09651080477494f5d1bf1d155857`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:07.029166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66c957ec-3073-434f-bd19-32cf301cbf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.180898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 888/1788] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:07.185415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.265357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:07.285540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc66de51-9892-4527-bf7e-9d6033a0d945 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.292764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.293489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.293895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.294179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.295022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.297244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b758-4e1b-46fd-968a-1822befea708 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.297754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.298076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b758-4e1b-46fd-968a-1822befea708 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.298641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7b758-4e1b-46fd-968a-1822befea708 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.301621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 889/1789] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:25:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:25:07.311003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.311558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.311921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.338673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] MFA Rules not processed for user `319c09d5d4db4a18bf1191375252c541`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:07.341199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.354089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 901/1790] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:07 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:07.355124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.451353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 890/1791] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:25:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:07.452341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.453749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.468250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e91a1-4e52-4786-88c6-cca4f88e87e2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.474292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.474644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.474930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.475204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] PATH_INFO: `/v3/users/c7702ef2d8c240a1b10bce8590ef495e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.476459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7702ef2d8c240a1b10bce8590ef495e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.477894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe79cd-0da6-48ab-bbb6-8e5254b81887 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.478402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.483703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.484428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.484714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.484995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.485707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.487787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.511394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 902/1792] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:07 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:07.512607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.532464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:07.535089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7702ef2d8c240a1b10bce8590ef495e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:07.536902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7702ef2d8c240a1b10bce8590ef495e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:07.547621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7702ef2d8c240a1b10bce8590ef495e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:07.564002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7702ef2d8c240a1b10bce8590ef495e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:07.573768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7702ef2d8c240a1b10bce8590ef495e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:07.583977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 891/1793] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:07 2022] DELETE /identity/v3/users/c7702ef2d8c240a1b10bce8590ef495e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:07.584878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.586299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.618735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b31ec8-3e0b-4907-aca2-fabf84d46997 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.674855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f78861-4536-4e2e-a7f0-dfc39137e212 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.681818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.682415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.682704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.682986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.683796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.685807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.707125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.712091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 892/1794] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:07.712171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.712171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.712323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.712388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.714103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.723493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ad115-28df-4d0a-8701-b1c45faae93e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.727289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.728551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.728901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.729595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.729941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.730180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] PATH_INFO: `/v3/projects/8606973e00ef47ada9620f1ac0e538d5/users/0db91a69ae34470e832e1586a1a0a120/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.733688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:07.743757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 903/1795] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:07.752351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8606973e00ef47ada9620f1ac0e538d5, user_id=0db91a69ae34470e832e1586a1a0a120, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.757165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.783145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 893/1796] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:07 2022] PUT /identity/v3/projects/8606973e00ef47ada9620f1ac0e538d5/users/0db91a69ae34470e832e1586a1a0a120/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:07.783392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.797053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbfba0-3828-47c9-bd37-747737768981 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.875195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.876054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.876351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.877196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.879014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.884874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.909815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.910545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 904/1797] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:07.922806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cad58-4f0d-431d-8db3-61bb076438eb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:07.928382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:07.929394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.929742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.930093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] PATH_INFO: `/v3/projects/8606973e00ef47ada9620f1ac0e538d5/users/0db91a69ae34470e832e1586a1a0a120/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.932301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8606973e00ef47ada9620f1ac0e538d5, user_id=0db91a69ae34470e832e1586a1a0a120, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:07.936192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:07.937272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3199c-7f02-49fc-a7d2-8f118df15e34 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:07.945084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:07.955289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 894/1798] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:07 2022] PUT /identity/v3/projects/8606973e00ef47ada9620f1ac0e538d5/users/0db91a69ae34470e832e1586a1a0a120/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:07.966208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202387-7562-44f0-a189-a70c095deca6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:07.966422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202387-7562-44f0-a189-a70c095deca6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:07.966771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79202387-7562-44f0-a189-a70c095deca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:07.976476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f69fa-db7e-4050-a8a6-3801b3665f40 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:08.001397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-79202387-7562-44f0-a189-a70c095deca6 None None] MFA Rules not processed for user `0db91a69ae34470e832e1586a1a0a120`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:08.002686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-79202387-7562-44f0-a189-a70c095deca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.062458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:08.063769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.064399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.064903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] PATH_INFO: `/v3/projects/f555cf9f335a4d48a15d2709b857a6d1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.066254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f555cf9f335a4d48a15d2709b857a6d1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:08.068410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:08.106320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 895/1799] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:08.119451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202387-7562-44f0-a189-a70c095deca6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:08.120943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-79202387-7562-44f0-a189-a70c095deca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.127681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f555cf9f335a4d48a15d2709b857a6d1', 'request_id': 'req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.131929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:08.135968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f555cf9f335a4d48a15d2709b857a6d1', 'request_id': 'req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.143805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f555cf9f335a4d48a15d2709b857a6d1', 'request_id': 'req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.164320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] The token cache is being invalidate because project f555cf9f335a4d48a15d2709b857a6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:08.164320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.164320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.166288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 905/1800] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:07 2022] DELETE /identity/v3/projects/f555cf9f335a4d48a15d2709b857a6d1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:08.169811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:08.171143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.189702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79202387-7562-44f0-a189-a70c095deca6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:08.198676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:08.251686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:08.251686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.251686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.251686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.252260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.254459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e92b1-c9bf-4ee2-9852-b402c7a61fbe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:08.260560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:08.261460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.261746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.262030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] PATH_INFO: `/v3/users/27251177bf404a258bdb5464508534ce` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.263259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=27251177bf404a258bdb5464508534ce)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:08.265173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:08.314849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:08.315045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27251177bf404a258bdb5464508534ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.316865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:08.327303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 896/1801] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:08.328681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27251177bf404a258bdb5464508534ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.339630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27251177bf404a258bdb5464508534ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.350769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27251177bf404a258bdb5464508534ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.351869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27251177bf404a258bdb5464508534ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.361438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 906/1802] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:08 2022] DELETE /identity/v3/users/27251177bf404a258bdb5464508534ce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:08.413652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:08.419912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.453278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d8658-867b-4960-a149-842a0b389d83 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:08.521043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:08.521520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.521790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.522060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.524262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.557298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:08.601564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:08.603049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:08.612176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:08.613620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88722e52-1424-4ea3-b332-ddadc3c7887c service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:08.625958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 897/1803] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:08.764848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac44c1-03af-4cec-8aae-33cb690ad8eb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:08.770565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:08.771572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.771823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.772194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] PATH_INFO: `/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.774912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8747dbebeebe4b599d4d08ce9c40c4d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:08.777046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:08.860799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:08.883774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8747dbebeebe4b599d4d08ce9c40c4d9', 'request_id': 'req-c2346afa-c819-433e-8457-21dcd6194e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.905248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8747dbebeebe4b599d4d08ce9c40c4d9', 'request_id': 'req-c2346afa-c819-433e-8457-21dcd6194e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.923424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8747dbebeebe4b599d4d08ce9c40c4d9', 'request_id': 'req-c2346afa-c819-433e-8457-21dcd6194e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.941034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] The token cache is being invalidate because project 8747dbebeebe4b599d4d08ce9c40c4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:08.941264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.941928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2346afa-c819-433e-8457-21dcd6194e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:08.943947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 907/1804] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:08 2022] DELETE /identity/v3/projects/8747dbebeebe4b599d4d08ce9c40c4d9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:08.948669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:08.948985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:08.949252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:08.971676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:08.973221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:09.106276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 898/1805] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:09.114697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.115049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.115457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.143652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:09.148728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:09.164237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 908/1806] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:09.169676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.185883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a0383-f141-4d4b-8e7d-a8f685e8818d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.195650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.196350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.196638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.196922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.199967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:09.201202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.203689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.268870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 899/1807] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:09.270563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.283327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cb59d-3538-4ce4-b26c-b308b2afeb2e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.299747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.300861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.301243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.301400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.302571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.304986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.341856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 909/1808] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:09.350579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.364098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb7cce-15ad-4470-a370-4c63c8a7d099 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.430340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.431124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.431496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.433179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.433179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.434929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.463306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 900/1809] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:09.466809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.479834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d16cb4-ad3e-4033-ba1c-24c064a0b642 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.488258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.488927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.489415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.489808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.490681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.492472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.504438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 910/1810] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:09.508279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.520619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c21eb-37de-494e-8496-6c309b2405e4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.525774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.526525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.526896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.527306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.528506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.530808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade21a77-3faf-4aab-ba54-a4a0f5740312 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.544902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 901/1811] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:09.549963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.565848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50c01c-72b2-49b5-9db5-d2e2199f2ab3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.570609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.571549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.571865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.572152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] PATH_INFO: `/v3/projects/e865927b70994f63b22fc84d34b79321/users/e604bdc64bd043689fed16c5c4302d45/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.587979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e865927b70994f63b22fc84d34b79321, user_id=e604bdc64bd043689fed16c5c4302d45, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.591392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.608715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 911/1812] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:09 2022] PUT /identity/v3/projects/e865927b70994f63b22fc84d34b79321/users/e604bdc64bd043689fed16c5c4302d45/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:09.620667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.621224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.621396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.652100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] MFA Rules not processed for user `e604bdc64bd043689fed16c5c4302d45`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:09.653324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:09.674171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:09.731790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 902/1813] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:09.736695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.804108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a028e-361a-47f5-a8be-dd6c40a18463 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.808757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.809427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.809717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.809999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.810699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.812671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.841174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 912/1814] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:09.844725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.856333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f3f0d-bd52-4fdd-8fe5-27092b6c3135 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.917933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.918639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.918928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.919209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.919984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.921960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.947080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 903/1815] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:09 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:09.951252 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:09.964296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d321-6d99-45b5-8788-ea3d22923b0e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:09.969195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:09.969978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:09.970395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:09.970752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:09.971614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:09.973395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:09.999257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 913/1816] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:10.003982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:10.014459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823b4ff-23a9-4a83-a26b-53926e37c580 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:10.032093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:10.033314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.033599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.033878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] PATH_INFO: `/v3/projects/40963b4365e547fa87404c31cb2bf681/users/9bdcd9593a0d41a78faf09b2d68ea135/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.059582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40963b4365e547fa87404c31cb2bf681, user_id=9bdcd9593a0d41a78faf09b2d68ea135, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:10.062325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:10.082414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:10.083748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 904/1817] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:10 2022] PUT /identity/v3/projects/40963b4365e547fa87404c31cb2bf681/users/9bdcd9593a0d41a78faf09b2d68ea135/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:10.095196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032723-7c9c-4258-b5d4-056d6f05997d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:10.153615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:10.171849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:10.172534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.172820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.173101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.173869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:10.175511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95084455-fb8d-4202-9c6f-5d6e2f46eb9e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:10.185395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 914/1818] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:10.190203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:10.200941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a4ca4-31f9-4eed-934b-5793cb802328 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:10.206533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:10.207737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.208021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.208451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] PATH_INFO: `/v3/projects/40963b4365e547fa87404c31cb2bf681/users/9bdcd9593a0d41a78faf09b2d68ea135/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.210483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40963b4365e547fa87404c31cb2bf681, user_id=9bdcd9593a0d41a78faf09b2d68ea135, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:10.213233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:10.224713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 905/1819] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:10 2022] PUT /identity/v3/projects/40963b4365e547fa87404c31cb2bf681/users/9bdcd9593a0d41a78faf09b2d68ea135/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:10.237745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.238442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.238824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.272071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] MFA Rules not processed for user `9bdcd9593a0d41a78faf09b2d68ea135`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:10.273320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:10.425641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 915/1820] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:10.436891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:10.443725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:10.532029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf08c4-4bba-4e23-8153-78dcc9fcb142 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:10.536035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:10.536698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.536983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.537262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.548233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:10.550560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ef1ac-62f2-4c85-ba4b-2fcec694359e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:10.561589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 906/1821] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:10.743583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:10.865249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c3cec-5263-4bde-8521-faa833ac66fc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:10.880687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:10.881888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:10.882240 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:10.883762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] PATH_INFO: `/v3/users/e604bdc64bd043689fed16c5c4302d45` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:10.885178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] RBAC: Authorizing `identity:delete_user(user_id=e604bdc64bd043689fed16c5c4302d45)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:10.887597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:10.975462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e604bdc64bd043689fed16c5c4302d45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:10.994270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e604bdc64bd043689fed16c5c4302d45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:10.998933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:11.009769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e604bdc64bd043689fed16c5c4302d45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.021109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e604bdc64bd043689fed16c5c4302d45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.021983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e604bdc64bd043689fed16c5c4302d45'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.036299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:11.036869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:11.037084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:11.038594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 916/1822] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:10 2022] DELETE /identity/v3/users/e604bdc64bd043689fed16c5c4302d45 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:11.060890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:11.062398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:11.078240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:11.134899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 907/1823] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:11.146726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:11.148941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:11.224674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971496a3-adb9-4fa6-80fb-1fd6017005db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:11.245130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:11.245742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:11.246026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:11.246314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:11.265692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:11.268152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:11.278339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 917/1824] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:11.752903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:11.754766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:11.786706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569422d6-cf55-4fe6-b003-282e8d63bf8b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:11.797591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:11.798992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:11.799345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:11.799662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] PATH_INFO: `/v3/projects/e865927b70994f63b22fc84d34b79321` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:11.800938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e865927b70994f63b22fc84d34b79321)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:11.802895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:11.872062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e865927b70994f63b22fc84d34b79321', 'request_id': 'req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.882210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e865927b70994f63b22fc84d34b79321', 'request_id': 'req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.890079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e865927b70994f63b22fc84d34b79321', 'request_id': 'req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.902873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] The token cache is being invalidate because project e865927b70994f63b22fc84d34b79321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:11.903444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.904284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:11.906045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 908/1825] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:11 2022] DELETE /identity/v3/projects/e865927b70994f63b22fc84d34b79321 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:11.910371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:11.928279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38620c9-7524-4ea9-875c-05c467985a10 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:12.041865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:12.042829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:12.043178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:12.043721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] PATH_INFO: `/v3/users/9bdcd9593a0d41a78faf09b2d68ea135` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:12.045051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bdcd9593a0d41a78faf09b2d68ea135)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:12.048161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:12.105844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bdcd9593a0d41a78faf09b2d68ea135'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:12.121909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bdcd9593a0d41a78faf09b2d68ea135'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:12.133345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bdcd9593a0d41a78faf09b2d68ea135'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:12.143540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bdcd9593a0d41a78faf09b2d68ea135'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:12.144502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bdcd9593a0d41a78faf09b2d68ea135'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:12.154576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 918/1826] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:11 2022] DELETE /identity/v3/users/9bdcd9593a0d41a78faf09b2d68ea135 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:12.774184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:12.775809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:12.798843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fbf6-e2bc-4575-86b4-5c0520a183a3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:12.871119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:12.872676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:12.872676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:12.872676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] PATH_INFO: `/v3/projects/40963b4365e547fa87404c31cb2bf681` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:12.875950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40963b4365e547fa87404c31cb2bf681)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:12.879032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:12.892776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:12.894482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:12.914215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3ef5b-b42c-4634-9636-c8605dea284b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:12.927495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:12.927495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:12.927495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:12.931555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] PATH_INFO: `/v3/users/3c94a216fd174ff08dbd798d986f1f31` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:12.934699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c94a216fd174ff08dbd798d986f1f31)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:12.939010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:12.942574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:12.986707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:12.998275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40963b4365e547fa87404c31cb2bf681', 'request_id': 'req-22a7fd0e-4310-4edf-aa4a-2ac292a02813'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.010777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40963b4365e547fa87404c31cb2bf681', 'request_id': 'req-22a7fd0e-4310-4edf-aa4a-2ac292a02813'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.021667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40963b4365e547fa87404c31cb2bf681', 'request_id': 'req-22a7fd0e-4310-4edf-aa4a-2ac292a02813'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.045504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c94a216fd174ff08dbd798d986f1f31'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.054881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] The token cache is being invalidate because project 40963b4365e547fa87404c31cb2bf681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:13.055339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.055950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-22a7fd0e-4310-4edf-aa4a-2ac292a02813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.064063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c94a216fd174ff08dbd798d986f1f31'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.068142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 909/1827] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:12 2022] DELETE /identity/v3/projects/40963b4365e547fa87404c31cb2bf681 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:13.073120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578a00-f085-4751-a994-a7a5d284a15a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.073625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c94a216fd174ff08dbd798d986f1f31'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.074130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578a00-f085-4751-a994-a7a5d284a15a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.075623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3578a00-f085-4751-a994-a7a5d284a15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.104095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c94a216fd174ff08dbd798d986f1f31'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.105260 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c94a216fd174ff08dbd798d986f1f31'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.118140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d3578a00-f085-4751-a994-a7a5d284a15a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:13.119391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d3578a00-f085-4751-a994-a7a5d284a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.125687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 919/1828] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:12 2022] DELETE /identity/v3/users/3c94a216fd174ff08dbd798d986f1f31 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:13.126995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.129192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.219414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db63951-3a71-475d-ad85-fac7b5e32211 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.242065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 910/1829] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:13.242470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.243943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.245187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.245592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] PATH_INFO: `/v3/users/5a8f09651080477494f5d1bf1d155857` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.251886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaf00-e1a2-4eb8-b7b1-7aa3ee404419 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.252526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaf00-e1a2-4eb8-b7b1-7aa3ee404419 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.253026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaf00-e1a2-4eb8-b7b1-7aa3ee404419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.254874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8f09651080477494f5d1bf1d155857)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.256954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.289880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8dcbaf00-e1a2-4eb8-b7b1-7aa3ee404419 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:13.291558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8dcbaf00-e1a2-4eb8-b7b1-7aa3ee404419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.301229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 911/1830] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:13.302260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.302727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.303008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.348508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:13.355434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8f09651080477494f5d1bf1d155857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.355849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.369966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 912/1831] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:13.377874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.380698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8f09651080477494f5d1bf1d155857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.383293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.409624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8f09651080477494f5d1bf1d155857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.425232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8f09651080477494f5d1bf1d155857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.426158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8f09651080477494f5d1bf1d155857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:13.429703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:13.444646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 920/1832] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:13 2022] DELETE /identity/v3/users/5a8f09651080477494f5d1bf1d155857 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:13.447150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.451688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.481920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79e538-39c2-4323-80ca-5251faa7c67b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.543939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.544705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.545088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.545441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.556511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.579467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b10369-7169-48b0-b0a1-aad5a3b6933d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.595734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.596857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.597398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.598185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.599257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:13.599607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.601399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067dca0-a316-4d45-a5e6-e40efcadff77 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.604970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:13.628305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 921/1833] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:13.630348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.630886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.631434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.650709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:13.652427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.661769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 922/1834] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:13.662840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.678179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.679922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.680988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb1cef-ba51-4087-8c34-85abb3f42b82 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.692069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.693076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.693550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.694013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.694894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.702106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.705550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 913/1835] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:13.706854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.716878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:13.736221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d3c33-34fa-4893-8a62-e38585e1f6b2 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.741692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.748026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.748342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.748804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.759183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 923/1836] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:13.762818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:13.771527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.774621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.812712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 914/1837] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:13.888626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0ebf8-8799-4bc1-95df-ee2d0d34a522 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:13.903031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:13.904125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:13.904610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:13.905092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:13.906360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:13.914269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:13.994339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 924/1838] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:13 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:13.997458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.009673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed012a0-76c3-4aa3-8f86-90e8d3d2cfc3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:14.028276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:14.030458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:14.030951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:14.031874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:14.034646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:14.037820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:14.093588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 915/1839] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:14.098284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.119297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011ca4b-4e50-40dc-86ec-a2954e7d40f0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:14.126357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:14.127329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:14.127830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:14.128318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:14.129278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:14.132466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be136545-03d0-4608-84bd-dd5f8fa2e06a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:14.166842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 925/1840] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:14.169866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.189157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3500f26-065d-4837-84a4-cb040567e17a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:14.205955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:14.206984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:14.207361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:14.207716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] PATH_INFO: `/v3/projects/3fe410d6750b4127a1c659d8e6b53e35/users/a4233ce0dbdd4045abdce72012ba7f5f/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:14.235864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe410d6750b4127a1c659d8e6b53e35, user_id=a4233ce0dbdd4045abdce72012ba7f5f, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:14.254826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:14.281099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 916/1841] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:14 2022] PUT /identity/v3/projects/3fe410d6750b4127a1c659d8e6b53e35/users/a4233ce0dbdd4045abdce72012ba7f5f/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:14.293553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590821-7302-4453-b58f-4344a64f681f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:14.294061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590821-7302-4453-b58f-4344a64f681f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:14.294354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3590821-7302-4453-b58f-4344a64f681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:14.362288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a3590821-7302-4453-b58f-4344a64f681f None None] MFA Rules not processed for user `a4233ce0dbdd4045abdce72012ba7f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:14.366106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a3590821-7302-4453-b58f-4344a64f681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:14.443662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:14.552949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 926/1842] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:14.557669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.764516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de35103-ed84-430e-ac56-02a37c6ba0d0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:14.780343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:14.786207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:14.790860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:14.792920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:14.793025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:14.817645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:14.867987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590821-7302-4453-b58f-4344a64f681f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.873301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 917/1843] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:14 2022] POST /identity/v3/projects => generated 379 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:14.885274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3590821-7302-4453-b58f-4344a64f681f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:14.893053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:14.895766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.037142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1d465-7be1-4974-aea7-11dc47b3fd9a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.043340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.046544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.047008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.047471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.047814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.048959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.049695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.050092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.050377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] PATH_INFO: `/v3/projects/b0d1cb88d7d247f58669117081033ea9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.051826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.051994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d1cb88d7d247f58669117081033ea9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.054392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.057362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:15.082324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 927/1844] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:14 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:15.083127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.084925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.099254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d83fee-9141-4b7d-a0e8-6399d5341eb5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.104845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.105857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.106343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.107091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] PATH_INFO: `/v3/projects/ebc86c40e47c4945976ff6dd7048b49f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.108160 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc86c40e47c4945976ff6dd7048b49f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.110204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.155000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d1cb88d7d247f58669117081033ea9', 'request_id': 'req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.165040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d1cb88d7d247f58669117081033ea9', 'request_id': 'req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.182402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc86c40e47c4945976ff6dd7048b49f', 'request_id': 'req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.192278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc86c40e47c4945976ff6dd7048b49f', 'request_id': 'req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.193014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d1cb88d7d247f58669117081033ea9', 'request_id': 'req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.200420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc86c40e47c4945976ff6dd7048b49f', 'request_id': 'req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.222557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] The token cache is being invalidate because project b0d1cb88d7d247f58669117081033ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:15.223637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.224416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.224828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] The token cache is being invalidate because project ebc86c40e47c4945976ff6dd7048b49f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:15.225131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.225809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2a9d7e4e-30a4-49a8-91ba-384d05e63ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.230521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 928/1846] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:15 2022] DELETE /identity/v3/projects/ebc86c40e47c4945976ff6dd7048b49f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:15.230690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 918/1846] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:14 2022] DELETE /identity/v3/projects/b0d1cb88d7d247f58669117081033ea9 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:15.232348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.233994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.234539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.235034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.235506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.264864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:15.266445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.317869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:15.320872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f87364-ae72-4db9-9a4f-b8d8482e7ec6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.347384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 929/1847] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:15.348570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.349977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.362911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6ccfc-f49d-437d-8d42-77ced19dd7c2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.371647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.372557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.372836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.373356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] PATH_INFO: `/v3/users/0db91a69ae34470e832e1586a1a0a120` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.374545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db91a69ae34470e832e1586a1a0a120)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.377086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.428328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.428741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.430482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.431485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.432325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.434269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e07595-c9f0-4260-996e-b0f69efb0b26 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.453847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 919/1848] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:15.456958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.457260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.457545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.476677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db91a69ae34470e832e1586a1a0a120'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.506951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db91a69ae34470e832e1586a1a0a120'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.507387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:15.509475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.523929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db91a69ae34470e832e1586a1a0a120'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.535127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 920/1849] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:15.535127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.537999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db91a69ae34470e832e1586a1a0a120'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.538302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.538844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db91a69ae34470e832e1586a1a0a120'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:15.553803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 930/1850] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:15 2022] DELETE /identity/v3/users/0db91a69ae34470e832e1586a1a0a120 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:15.559932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.561925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.562628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51793bc9-77cb-4b9d-b2b4-ff699c3452fd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.618645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:15.688348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.689610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.690195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.690195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.690810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f07b-314c-4111-be0a-367e17774c00 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.690935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.692676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.696954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.697793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.698083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.698374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.699325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:15.700888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.701500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.707413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 921/1851] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:15.709354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.714603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:15.723254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 931/1852] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:15.727355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.742814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae119166-e737-48b4-86c3-7ddd0d2a5d13 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.752610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad318893-0bfb-439d-bebc-7f6ccee3af18 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:15.765167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.766122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.766394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.766738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] PATH_INFO: `/v3/projects/e825ce290bdc495b9073dad51e45da10/users/b7a7d30dc86d408dabaa89ea2f523978/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.768310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:15.769063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.769849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.770149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.771025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.774705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.820364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 932/1853] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:15 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:15.824558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:15.831269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e825ce290bdc495b9073dad51e45da10, user_id=b7a7d30dc86d408dabaa89ea2f523978, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:15.833957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49784b50-7431-448c-9937-763ee9455c5b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:15.887938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 922/1854] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:15 2022] PUT /identity/v3/projects/e825ce290bdc495b9073dad51e45da10/users/b7a7d30dc86d408dabaa89ea2f523978/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:15.900514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:15.900809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:15.901093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:15.931423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:15.964028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] MFA Rules not processed for user `b7a7d30dc86d408dabaa89ea2f523978`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:15.965299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:16.066385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630508ac-30a1-4151-8542-1afd5a004073 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.105028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 923/1855] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:16.110268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.261758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ea9ef-0297-45db-aad1-6d10a7987e65 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.265874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.270221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.271185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.271409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.271836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.272318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.273166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.278812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.284003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.284371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.294758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.315353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.319409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 924/1856] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:16 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:16.323347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.357560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:16.383576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 933/1857] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:15 2022] POST /identity/v3/users => generated 289 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:16.389967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.415505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc044ed0-055d-4848-9c8c-a0ba36725df8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.434511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9885ae-3f95-4b4c-8c32-3c6dc773dc51 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.441291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.442315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.442880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.443298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.444114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.446589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.484585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 925/1858] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:16 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:16.486845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.521106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293146d1-ec60-4038-a612-3e3001ee60c5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.548199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.548963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.549258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.549514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.554841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.556653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.570956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 926/1859] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:16.571710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.573905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:16.589732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc668b4-254e-421d-81f2-7a5b000fe328 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.599907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.599907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.599907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.599907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] PATH_INFO: `/v3/projects/8606973e00ef47ada9620f1ac0e538d5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.601618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8606973e00ef47ada9620f1ac0e538d5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.604047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.630921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.632087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.632661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.633145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.641537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.645723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.668886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 934/1860] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:16.669808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.682608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41d269-f721-4d27-8e8b-2ae36108ad5d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.696027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8606973e00ef47ada9620f1ac0e538d5', 'request_id': 'req-c07528ad-45e1-45e3-858c-a99f4265d865'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:16.708589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8606973e00ef47ada9620f1ac0e538d5', 'request_id': 'req-c07528ad-45e1-45e3-858c-a99f4265d865'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:16.718401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8606973e00ef47ada9620f1ac0e538d5', 'request_id': 'req-c07528ad-45e1-45e3-858c-a99f4265d865'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:16.737114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] The token cache is being invalidate because project 8606973e00ef47ada9620f1ac0e538d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:16.737850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:16.739794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:16.742479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 927/1861] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:16 2022] DELETE /identity/v3/projects/8606973e00ef47ada9620f1ac0e538d5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:16.744204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.746452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:16.751925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.753106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.753456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.753837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] PATH_INFO: `/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17/users/ea6efd1e0e524028893ba5e204eefc6c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.758085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:16.765018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07528ad-45e1-45e3-858c-a99f4265d865 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.775139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af2e175fa6479ba7586cd10bbf6d17, user_id=ea6efd1e0e524028893ba5e204eefc6c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.775744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.776445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.776740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.777036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.777764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.779352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.779807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a97ea-efa1-4b85-a7a9-352eee9a9675 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.797071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 928/1862] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:16.799187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.799508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.799791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.803808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 935/1863] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:16 2022] PUT /identity/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17/users/ea6efd1e0e524028893ba5e204eefc6c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:16.804581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.824732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:16.827319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:16.935928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45de59-f45b-4c3f-b315-5f7e80eb292c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.947924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.949276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.949566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.949842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] PATH_INFO: `/v3/projects/d1251bd6ebe64bf4b1c0812da0400a05/users/14de3d1e5a1841389bc8ccabde1e73d1/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.957826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 929/1864] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:16.964636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:16.968777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:16.984098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186e90-907d-4ca6-abc5-4f291bf8c27d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:16.989947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1251bd6ebe64bf4b1c0812da0400a05, user_id=14de3d1e5a1841389bc8ccabde1e73d1, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.990447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:16.991341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:16.991644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:16.992974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:16.993078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.993184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:16.994612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d386c-f192-47d4-bd1c-16cef42123b1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:16.997394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:17.025534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 930/1866] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:17.027833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe864d8-b6d6-4e2c-9ccc-ad1272a147ca None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.028132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe864d8-b6d6-4e2c-9ccc-ad1272a147ca None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.029973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe864d8-b6d6-4e2c-9ccc-ad1272a147ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.031285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 936/1866] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:16 2022] PUT /identity/v3/projects/d1251bd6ebe64bf4b1c0812da0400a05/users/14de3d1e5a1841389bc8ccabde1e73d1/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:17.033619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.076390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cbe864d8-b6d6-4e2c-9ccc-ad1272a147ca None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:17.077683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cbe864d8-b6d6-4e2c-9ccc-ad1272a147ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:17.180356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 931/1867] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:17.186646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801bd0-6dce-4d83-9c17-c5442ec45f58 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.186646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801bd0-6dce-4d83-9c17-c5442ec45f58 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.187463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f801bd0-6dce-4d83-9c17-c5442ec45f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.226245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caf1ec-affe-466f-8f2f-e8339b599b8a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.232023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.233078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.233370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.233648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] PATH_INFO: `/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17/users/ea6efd1e0e524028893ba5e204eefc6c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.236369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af2e175fa6479ba7586cd10bbf6d17, user_id=ea6efd1e0e524028893ba5e204eefc6c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.245510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.245809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0f801bd0-6dce-4d83-9c17-c5442ec45f58 None None] MFA Rules not processed for user `14de3d1e5a1841389bc8ccabde1e73d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:17.247136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0f801bd0-6dce-4d83-9c17-c5442ec45f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:17.270304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 937/1868] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:17 2022] PUT /identity/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17/users/ea6efd1e0e524028893ba5e204eefc6c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:17.271430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.360344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 932/1869] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:17.367876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.368951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.368951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.405336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] MFA Rules not processed for user `ea6efd1e0e524028893ba5e204eefc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:17.408484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:17.411741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6f36d-9caf-4a4c-b5d1-eda5c466f0a9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.421426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.422696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.422987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.423697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.424112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:17.424423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.426319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.429137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:17.443567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 938/1870] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:17.443567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.449313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:17.512835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 933/1871] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:17.513745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.528034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601a5ef-f227-4ab2-b35c-84390e47f9b9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.535399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.536779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.537363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.537857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.538903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.542768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.554737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:17.570709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a533379-e1b8-44ed-a30e-179811e115c0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.576877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.578511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.579056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.580461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.592164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 934/1872] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:17 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:17.592644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.687776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.689749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.705438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 939/1873] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:17.728666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3cd6-61b6-4170-83ff-2ee8476e890c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.735102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.736437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.737242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.737242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.737928 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.740114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.769752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 935/1874] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:17 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:17.774471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.791752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258f829-35ae-47da-a373-92c426c4bdb4 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.808377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.812863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.813949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.814696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.817292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.820366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.849056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 940/1875] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:17.853762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.875347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c29e73-771c-4d91-b411-2e0e01c940d7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.895835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.897039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.897375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.898635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.898995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.899415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:17.903853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:17.906358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:17.940254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 936/1876] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:17.941218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:17.972834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63998600-0b13-48e0-a92b-d68436d5ae08 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:17.985300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:17.987123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:17.987892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:17.988507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] PATH_INFO: `/v3/projects/788f99e1f77946fe9f0771ffff9e934c/users/0deceba75df8425380687501594c6e0d/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:17.992498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae1450-0444-4e88-bbde-a42f35bfd0d9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.016693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788f99e1f77946fe9f0771ffff9e934c, user_id=0deceba75df8425380687501594c6e0d, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.019869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.020693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0410d-3f6e-4f73-bda2-5a2ae34ed563 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.021594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.022128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.023268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.060442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 937/1877] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:17 2022] PUT /identity/v3/projects/788f99e1f77946fe9f0771ffff9e934c/users/0deceba75df8425380687501594c6e0d/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:18.074464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.074787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.075036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.128024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:18.131228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] MFA Rules not processed for user `0deceba75df8425380687501594c6e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:18.134825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:18.148591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.153079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.180043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 941/1878] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:18.351060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 938/1879] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:18.356228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.375453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1146-dad6-479b-83f7-72f6e62b2f49 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.405435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.465659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:18.465825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.466405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.466693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.466982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.467719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.469671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.495737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63553835-8165-4462-a237-344a6b2a43b2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.500501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.501289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.501552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.501840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.504261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:18.505615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 942/1880] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:18 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:18.506520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.580801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1afef2-f7e6-49a0-b8b5-0774737212bc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.586526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.587565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.587901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.588171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.589809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.590208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.591503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.593078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.609365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 939/1881] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:18.619918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 943/1882] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:18 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:18.620734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.640516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda1fc3-0374-427a-ab0a-d4f033427f07 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.647874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.648731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.649090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.649438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.651575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.652929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.670787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 940/1883] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:18.675342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.688827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ab3dc-42e7-46d5-b5f0-9a578c93cb0f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.693910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.695166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.695404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.695687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] PATH_INFO: `/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d/users/002a10f98a1b478abca92599b5a14893/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.725739 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b9e48c44c545a6b89e5bd66f407a1d, user_id=002a10f98a1b478abca92599b5a14893, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.729058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.756882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.765595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 944/1884] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:18 2022] PUT /identity/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d/users/002a10f98a1b478abca92599b5a14893/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:18.781487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bab3-7759-449e-9269-9cf6e1b947d1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.924273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.925282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.925790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.926269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.928448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.930527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70937b-7eb2-4b2e-83c1-37a8d486b23d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:18.944525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 941/1885] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:18.948979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:18.967124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b3186-c63d-4c4c-989a-fa4559c411a9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:18.979733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:18.981520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:18.982123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:18.982819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] PATH_INFO: `/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d/users/002a10f98a1b478abca92599b5a14893/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:18.987013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b9e48c44c545a6b89e5bd66f407a1d, user_id=002a10f98a1b478abca92599b5a14893, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:18.991955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:19.006555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 945/1886] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:18 2022] PUT /identity/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d/users/002a10f98a1b478abca92599b5a14893/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:19.018048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32dfe-8ae3-4c7d-aec5-e5cb17cb596f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:19.019493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32dfe-8ae3-4c7d-aec5-e5cb17cb596f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:19.020106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32dfe-8ae3-4c7d-aec5-e5cb17cb596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:19.076491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c5b32dfe-8ae3-4c7d-aec5-e5cb17cb596f None None] MFA Rules not processed for user `002a10f98a1b478abca92599b5a14893`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:19.077742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c5b32dfe-8ae3-4c7d-aec5-e5cb17cb596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:19.197979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:19.206196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 942/1887] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:19.219175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:19.253092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:19.286641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfaa9b-21cb-4809-836e-901cd88b2936 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:19.290002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:19.290661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:19.290974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:19.291285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:19.360034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:19.361506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e46e9-7993-40db-98cc-c940cdbdad62 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:19.371681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 946/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:28.838586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:28.839147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:28.839427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:28.841043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 943/1889] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:25:28.850719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:28.851267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:28.851525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:28.872500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:28.873907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:28.948055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 947/1890] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:25:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:28.960156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:28.961863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:28.979706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68c4f1-7a5b-4ab2-87b6-c82c11135390 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:28.986477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:28.987142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:28.987460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:28.987734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:28.997292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:28.998780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ef698-27bc-4a37-9c81-94a67e04509a service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:29.010779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 944/1891] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:29.236987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:29.331125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffb0b8-2c30-4eb1-8ed1-f831149e3795 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:29.338076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:29.339280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:29.339715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:29.340327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] PATH_INFO: `/v3/users/a4233ce0dbdd4045abdce72012ba7f5f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:29.342552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4233ce0dbdd4045abdce72012ba7f5f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:29.345803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:29.378382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:29.448638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4233ce0dbdd4045abdce72012ba7f5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:29.475124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4233ce0dbdd4045abdce72012ba7f5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:29.490399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4233ce0dbdd4045abdce72012ba7f5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:29.503050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4233ce0dbdd4045abdce72012ba7f5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:29.503939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4233ce0dbdd4045abdce72012ba7f5f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:29.515819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 948/1892] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:29 2022] DELETE /identity/v3/users/a4233ce0dbdd4045abdce72012ba7f5f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:29.517913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:29.518215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:29.518503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:29.537691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:29.539104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:29.622217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 945/1893] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:29.644703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:29.646239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:29.796742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29286-33dd-4b27-8cd4-e596ebfaeb0e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:29.820162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:29.820720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:29.821215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:29.821574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:29.865236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:29.872507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:29.892313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 949/1894] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:30.359856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:30.362546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:30.377064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fff508-9de4-41ff-aa98-93f117c9c417 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:30.392290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:30.394385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:30.395084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:30.407413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] PATH_INFO: `/v3/projects/3fe410d6750b4127a1c659d8e6b53e35` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:30.412067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe410d6750b4127a1c659d8e6b53e35)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:30.415951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:30.473952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe410d6750b4127a1c659d8e6b53e35', 'request_id': 'req-473a036c-63c3-4656-8c59-8333c480a900'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.481130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe410d6750b4127a1c659d8e6b53e35', 'request_id': 'req-473a036c-63c3-4656-8c59-8333c480a900'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.488487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe410d6750b4127a1c659d8e6b53e35', 'request_id': 'req-473a036c-63c3-4656-8c59-8333c480a900'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.499171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] The token cache is being invalidate because project 3fe410d6750b4127a1c659d8e6b53e35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:30.499494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.500140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.503361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 946/1895] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:30 2022] DELETE /identity/v3/projects/3fe410d6750b4127a1c659d8e6b53e35 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:30.506688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:30.517185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d7664-f684-4e96-b5d8-168cd265ff78 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:30.587490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:30.588131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:30.588425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:30.588768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] PATH_INFO: `/v3/users/b7a7d30dc86d408dabaa89ea2f523978` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:30.590163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a7d30dc86d408dabaa89ea2f523978)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:30.592400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:30.667251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:30.679318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a7d30dc86d408dabaa89ea2f523978'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.698046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a7d30dc86d408dabaa89ea2f523978'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.710737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a7d30dc86d408dabaa89ea2f523978'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.726824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a7d30dc86d408dabaa89ea2f523978'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.728487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a7d30dc86d408dabaa89ea2f523978'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:30.744697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 950/1896] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:30 2022] DELETE /identity/v3/users/b7a7d30dc86d408dabaa89ea2f523978 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:31.207726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:31.210602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.230036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a036c-63c3-4656-8c59-8333c480a900 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:31.333007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:31.333976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:31.334265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:31.334662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] PATH_INFO: `/v3/projects/e825ce290bdc495b9073dad51e45da10` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:31.335910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e825ce290bdc495b9073dad51e45da10)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:31.338477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:31.350510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:31.403304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e825ce290bdc495b9073dad51e45da10', 'request_id': 'req-1c37adf7-0289-4420-b305-abd349c85fee'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.417097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e825ce290bdc495b9073dad51e45da10', 'request_id': 'req-1c37adf7-0289-4420-b305-abd349c85fee'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.425234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e825ce290bdc495b9073dad51e45da10', 'request_id': 'req-1c37adf7-0289-4420-b305-abd349c85fee'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.433206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:31.434623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.440046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] The token cache is being invalidate because project e825ce290bdc495b9073dad51e45da10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:31.440487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.441154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.442720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 947/1897] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:31 2022] DELETE /identity/v3/projects/e825ce290bdc495b9073dad51e45da10 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:31.446689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:31.448357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.536302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e4f34-707b-45f9-bc06-20d6c663e5bf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:31.565816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37adf7-0289-4420-b305-abd349c85fee admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:31.575384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:31.576307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:31.576722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:31.577092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] PATH_INFO: `/v3/users/ea6efd1e0e524028893ba5e204eefc6c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:31.578568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6efd1e0e524028893ba5e204eefc6c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:31.580879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:31.592237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:31.592964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:31.593339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:31.593677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:31.596428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.642284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6efd1e0e524028893ba5e204eefc6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.643547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6efd1e0e524028893ba5e204eefc6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.654816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6efd1e0e524028893ba5e204eefc6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.680457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6efd1e0e524028893ba5e204eefc6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.685832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:31.687424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcae43-5e11-4ada-8402-b4b3aa0eb255 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:31.688079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6efd1e0e524028893ba5e204eefc6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.694725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 948/1898] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:31 2022] DELETE /identity/v3/users/ea6efd1e0e524028893ba5e204eefc6c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:31.695565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:31.696947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.701388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 951/1899] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:31.732293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:31.788351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d7079-8c48-426c-9710-6909ba42b1e9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:31.793546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:31.794672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:31.795538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:31.795961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] PATH_INFO: `/v3/users/df7bdd442f0f4355bb75c64409fff6bb` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:31.797364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7bdd442f0f4355bb75c64409fff6bb)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:31.799935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:31.868207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7bdd442f0f4355bb75c64409fff6bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.869051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7bdd442f0f4355bb75c64409fff6bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.878219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7bdd442f0f4355bb75c64409fff6bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.899568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7bdd442f0f4355bb75c64409fff6bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.910663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7bdd442f0f4355bb75c64409fff6bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:31.920493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 949/1900] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:31 2022] DELETE /identity/v3/users/df7bdd442f0f4355bb75c64409fff6bb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:31.925296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:31.925595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:31.925879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:31.959625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:31.960902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:31.994229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:32.154234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 952/1901] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:32.167906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.169435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:32.292287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc9977-f97d-4a8b-bb40-546d9c2e7c87 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.300845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.301615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.302004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.302367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.305866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.308535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:32.330346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19804d-486d-4698-8211-33b2595d6deb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.334715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.341163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a293e57-300a-4ee2-846e-081b0d275a32 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.354587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.356131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.356534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.356929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] PATH_INFO: `/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.361082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0af2e175fa6479ba7586cd10bbf6d17)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.366886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.380367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 950/1902] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:32.456221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0af2e175fa6479ba7586cd10bbf6d17', 'request_id': 'req-99b57022-03b8-4bd6-8977-5f8557f78b86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:32.463401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0af2e175fa6479ba7586cd10bbf6d17', 'request_id': 'req-99b57022-03b8-4bd6-8977-5f8557f78b86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:32.469331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0af2e175fa6479ba7586cd10bbf6d17', 'request_id': 'req-99b57022-03b8-4bd6-8977-5f8557f78b86'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:32.486164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] The token cache is being invalidate because project f0af2e175fa6479ba7586cd10bbf6d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:32.486164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:32.486472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-99b57022-03b8-4bd6-8977-5f8557f78b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:32.491262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 953/1903] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:32 2022] DELETE /identity/v3/projects/f0af2e175fa6479ba7586cd10bbf6d17 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:32.494470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.494970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.495059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.532087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:32.533363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:32.582469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:32.633608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 951/1904] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:32.633801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.634151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.634444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.666047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:32.667545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:32.685569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 954/1905] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:32.691299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.703269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cfba1d-5984-480c-98e4-8dc102b1eb7f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.709471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.710164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.711115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.711266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.712099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:32.712666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.714436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.738566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 952/1906] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:32.739455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.755543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea399a-1af1-4ac7-baf2-f0236c468438 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.763204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.764348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.764703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.765045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.765816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.768281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.809011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 955/1907] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:32 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:32.812954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.825894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.827181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802e29c-7178-498c-9c89-1ec080d3ccb8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.829075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:32.940789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.941754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.942444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.942886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.943724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.945957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.959275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b1375-4241-43e4-bbf4-a4401b41afa3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:32.964800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:32.965845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:32.966197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:32.966483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] PATH_INFO: `/v3/projects/2547c48c6bbd415aae27ed306f22ebbc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:32.972841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2547c48c6bbd415aae27ed306f22ebbc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:32.976595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 953/1908] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:32 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:32.978107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:32.979837 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:32.993712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e6f36-fae2-4947-9744-1fea6688f4e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.000315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.001748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.002142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.002766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.003811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.006889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.022544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 954/1909] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:33.026438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.032718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:33.046115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264cac0-cf2e-4866-a4ac-8dbedf12c14d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.064102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.065438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.065907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.066638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.067431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.069322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.087936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 955/1910] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:33.088713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.097325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2547c48c6bbd415aae27ed306f22ebbc', 'request_id': 'req-8853be59-d1a1-41c5-8ccb-2643d54ec836'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.099641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d39d6-fb9c-4d79-8798-d4df2a1cb045 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.104656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2547c48c6bbd415aae27ed306f22ebbc', 'request_id': 'req-8853be59-d1a1-41c5-8ccb-2643d54ec836'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.111267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2547c48c6bbd415aae27ed306f22ebbc', 'request_id': 'req-8853be59-d1a1-41c5-8ccb-2643d54ec836'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.125997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] The token cache is being invalidate because project 2547c48c6bbd415aae27ed306f22ebbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:33.126283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.126936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.128610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 956/1911] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:32 2022] DELETE /identity/v3/projects/2547c48c6bbd415aae27ed306f22ebbc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:33.132574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.133911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.165213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.166110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.166421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.166696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] PATH_INFO: `/v3/projects/3bc336c2911a4fb195affbc82ec4286a/users/7e0dd6aa095e4f7a9f858c822d23dfbf/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.182602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc336c2911a4fb195affbc82ec4286a, user_id=7e0dd6aa095e4f7a9f858c822d23dfbf, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.185328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54a56-52d9-46f9-9f66-4e4883ca7beb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.199626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 956/1912] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:33 2022] PUT /identity/v3/projects/3bc336c2911a4fb195affbc82ec4286a/users/7e0dd6aa095e4f7a9f858c822d23dfbf/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:33.200798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853be59-d1a1-41c5-8ccb-2643d54ec836 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.217080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.217396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.217664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.258445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] MFA Rules not processed for user `7e0dd6aa095e4f7a9f858c822d23dfbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:33.259737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.266369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:33.278669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.279789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.280064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.280343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] PATH_INFO: `/v3/users/8126a3e25e7e4df29f75b76d8f131e36` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.281609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8126a3e25e7e4df29f75b76d8f131e36)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.284534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.344253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8126a3e25e7e4df29f75b76d8f131e36'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.350727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:33.360505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8126a3e25e7e4df29f75b76d8f131e36'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.361136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 957/1913] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:33.365618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.368377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.377773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8126a3e25e7e4df29f75b76d8f131e36'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.384629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0193a-504e-434f-a19e-522cbb0e96df None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.386532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8126a3e25e7e4df29f75b76d8f131e36'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.387239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8126a3e25e7e4df29f75b76d8f131e36'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.389876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.390604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.390900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.391184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.392461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.395195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.395560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 957/1914] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:33 2022] DELETE /identity/v3/users/8126a3e25e7e4df29f75b76d8f131e36 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:33.436001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.437611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.439865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 958/1915] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:33 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:33.527731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5a9f3-390e-4a00-b39e-544bae75d1f5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.602926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.603678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.603968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.606998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.607781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.609930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.638150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 958/1916] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:33 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:33.643206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.654323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a91a6-574b-4431-a552-ba035077a4e2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.660388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.661076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.661369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.661654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.662579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.665488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.681381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 959/1917] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:33.684189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.695464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48eb4-002b-4d84-a5eb-eb35d94b6aae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.700914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.701871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.702227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.702511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] PATH_INFO: `/v3/projects/3a006ff187b9412bb5e46400e63c63e0/users/e959a80eb9c84665862a5d93e58df05c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.720145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a006ff187b9412bb5e46400e63c63e0, user_id=e959a80eb9c84665862a5d93e58df05c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.722846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.737903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 959/1918] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:33 2022] PUT /identity/v3/projects/3a006ff187b9412bb5e46400e63c63e0/users/e959a80eb9c84665862a5d93e58df05c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:33.742760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.754406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79933c24-d8ec-4c7e-a2e6-6fa2d9456eeb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.760693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.762404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.814400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:33.817380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.818185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.818639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.818925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.819797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.821536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.832678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 960/1919] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:33.836758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:33.837012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b282-a1ed-4d7a-93dc-4e057e3840af admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.841723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.842575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.842864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.843141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] PATH_INFO: `/v3/projects/b1040826386b48b9ac0974c0002cec9f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.844388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1040826386b48b9ac0974c0002cec9f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.846840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.846952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1c887-fe0c-4e26-8307-bdd7cc41cdb0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:33.851801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:33.852672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.852966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.853244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] PATH_INFO: `/v3/projects/3a006ff187b9412bb5e46400e63c63e0/users/e959a80eb9c84665862a5d93e58df05c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.855355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a006ff187b9412bb5e46400e63c63e0, user_id=e959a80eb9c84665862a5d93e58df05c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:33.858155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5bb3a-4cb7-4b06-aeb3-e2eef4177bd0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:33.869721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 961/1920] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:33 2022] PUT /identity/v3/projects/3a006ff187b9412bb5e46400e63c63e0/users/e959a80eb9c84665862a5d93e58df05c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:33.881418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.881963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.882233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.900414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1040826386b48b9ac0974c0002cec9f', 'request_id': 'req-856c0450-97b4-4add-84b3-4be1ec724a87'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.907709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1040826386b48b9ac0974c0002cec9f', 'request_id': 'req-856c0450-97b4-4add-84b3-4be1ec724a87'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.912171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] MFA Rules not processed for user `e959a80eb9c84665862a5d93e58df05c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:33.913582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.914390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1040826386b48b9ac0974c0002cec9f', 'request_id': 'req-856c0450-97b4-4add-84b3-4be1ec724a87'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.933412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] The token cache is being invalidate because project b1040826386b48b9ac0974c0002cec9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:33.934792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.936890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-856c0450-97b4-4add-84b3-4be1ec724a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:33.939545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 960/1921] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:33 2022] DELETE /identity/v3/projects/b1040826386b48b9ac0974c0002cec9f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:33.945768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518d364-a949-4922-93fb-129cdcccd868 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:33.946089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518d364-a949-4922-93fb-129cdcccd868 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:33.946633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518d364-a949-4922-93fb-129cdcccd868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:33.968892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6518d364-a949-4922-93fb-129cdcccd868 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:33.970758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6518d364-a949-4922-93fb-129cdcccd868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:33.972048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:33.999994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 962/1922] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:34.010982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.012902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.043268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 961/1923] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:34.052161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.052515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.052810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.074297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:34.075821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.085159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 962/1924] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:34.089409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.094691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d8d3-94ca-4d92-8f76-d81460bb8176 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.099236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.099426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921c4f5-8d28-452e-9c3f-2e7a2840bc2f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.099906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.100195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.100475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.104122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.104778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.105069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.105351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.106104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:34.106420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.108088 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.112232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.113909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.120560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 963/1925] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:25:34.124771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 963/1926] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:34.125779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.136535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ccf6-d20e-4458-bef4-182f001157e4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.141216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.142636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.143031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.143968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.144158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.146370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.174354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 964/1927] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:34.178690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.189871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d42-84b8-45aa-951f-3635ccc883aa service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.253902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.254608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.254865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.255175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.255932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.257920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.284110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 964/1928] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:34.285441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.296469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a7658-b9cd-4183-896f-4e7718661409 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.301468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.302224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.302575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.302938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.303812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.305409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.316248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 965/1929] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:34.320055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.330868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09526a33-bb36-46ad-ba66-1e3a6fdc7abe admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.335970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.340504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.340504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.340504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.340504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.340504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217de2dd-d9a0-4ec4-8962-5c468c3ae04d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.354189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 965/1930] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:34.355615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.366019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6627-5eef-4870-b7c5-3a195ae765c3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.371767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.372763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.373112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.373481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] PATH_INFO: `/v3/projects/87d903e3ff5446179523a1598cdb0342/users/36fec93a443d4be890257f144bdaa8d9/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.392827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d903e3ff5446179523a1598cdb0342, user_id=36fec93a443d4be890257f144bdaa8d9, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.392936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.402530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:34.409126 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 966/1931] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:34 2022] PUT /identity/v3/projects/87d903e3ff5446179523a1598cdb0342/users/36fec93a443d4be890257f144bdaa8d9/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:34.419459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.419824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.420177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.450548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] MFA Rules not processed for user `36fec93a443d4be890257f144bdaa8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:34.451903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.512447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:34.524153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 966/1932] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:34.528710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.579065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.581651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.599169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13e2d1-ad25-4e38-bcae-c588c53d9f15 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.604425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.605618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.606001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.606377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.607437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.609472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.640730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 967/1933] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:34.645062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.662799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00948e18-ddf8-4b59-a3ae-0491362c9f5c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.724214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de4d5a-32e0-488f-b722-9435a6741dd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.726447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.727373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.727735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.728026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] PATH_INFO: `/v3/users/7e0dd6aa095e4f7a9f858c822d23dfbf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.729230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0dd6aa095e4f7a9f858c822d23dfbf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.729334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.730041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.730572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.730893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.731165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.731785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.733996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.758002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 968/1934] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:34.762340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.774839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7b0a0-6e32-45ee-ac61-556d9427a71b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.780338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.781078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.781399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.781989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.782760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.784508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.791343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0dd6aa095e4f7a9f858c822d23dfbf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:34.792083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0dd6aa095e4f7a9f858c822d23dfbf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:34.796684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 969/1935] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:34.801446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.802351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0dd6aa095e4f7a9f858c822d23dfbf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:34.803499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.817312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5f79-7edf-4278-b736-e52af92a7816 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.819572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0dd6aa095e4f7a9f858c822d23dfbf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:34.822271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.823274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.823632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.823985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] PATH_INFO: `/v3/projects/b6693809e91a497097473bd90774d268/users/a8fd94c7188f4c068102c986f952ef6c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.831581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f7f10ae5-cfd5-4484-80ed-bba376602816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0dd6aa095e4f7a9f858c822d23dfbf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:34.838951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 967/1936] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:34 2022] DELETE /identity/v3/users/7e0dd6aa095e4f7a9f858c822d23dfbf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:34.840151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6693809e91a497097473bd90774d268, user_id=a8fd94c7188f4c068102c986f952ef6c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.843071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.843285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.843387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.843677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.859694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 970/1937] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:34 2022] PUT /identity/v3/projects/b6693809e91a497097473bd90774d268/users/a8fd94c7188f4c068102c986f952ef6c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:34.864353 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.864608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:34.865982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.883343 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:34.910311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:34.933591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c71d3-95aa-4845-9e7a-677aa9aab669 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.935578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 968/1938] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:34.938348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.939910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.940195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.940471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.941213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.942808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.945653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.946995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:34.956244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 971/1939] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:34.957265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:34.966917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745ce98-430b-4fa8-ab71-4da1d8459a58 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:34.971507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:34.972392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.972680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:34.972956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] PATH_INFO: `/v3/projects/b6693809e91a497097473bd90774d268/users/a8fd94c7188f4c068102c986f952ef6c/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:34.974939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6693809e91a497097473bd90774d268, user_id=a8fd94c7188f4c068102c986f952ef6c, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:34.977658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d66b06-21d7-421b-8f82-640abbd3b29e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:34.988350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 972/1940] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:34 2022] PUT /identity/v3/projects/b6693809e91a497097473bd90774d268/users/a8fd94c7188f4c068102c986f952ef6c/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:34.999497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:34.999899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.000188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.007636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac70f38-41d4-4449-916e-e5e46990f173 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.029129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] MFA Rules not processed for user `a8fd94c7188f4c068102c986f952ef6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:35.030349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.056911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.057510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.057799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.058081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.137803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 973/1941] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:35.150405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.151822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.151945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.153294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.170111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 969/1942] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:35.217265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe83c8-0fae-41aa-913b-5b1892ff328b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.221686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.222433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.222786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.223354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.233219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.234847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.246295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 974/1943] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:35.524868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.526625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.540897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d42a-0d27-4c0e-99bb-3c7193d26dd3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.546536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.547770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.547878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.548100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] PATH_INFO: `/v3/projects/3bc336c2911a4fb195affbc82ec4286a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.549347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc336c2911a4fb195affbc82ec4286a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.551482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.575851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:35.603855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc336c2911a4fb195affbc82ec4286a', 'request_id': 'req-ba096148-ee90-4430-a635-84bb91602c5d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.611874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc336c2911a4fb195affbc82ec4286a', 'request_id': 'req-ba096148-ee90-4430-a635-84bb91602c5d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.619035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc336c2911a4fb195affbc82ec4286a', 'request_id': 'req-ba096148-ee90-4430-a635-84bb91602c5d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.630153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] The token cache is being invalidate because project 3bc336c2911a4fb195affbc82ec4286a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:35.630450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.630789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.631276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.633174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 970/1944] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:35 2022] DELETE /identity/v3/projects/3bc336c2911a4fb195affbc82ec4286a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:35.636627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.637937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.640309 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe7437-f497-4d76-8ed9-871888fbde08 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.695568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:35.722120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba096148-ee90-4430-a635-84bb91602c5d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.726183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8fd94c7188f4c068102c986f952ef6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6693809e91a497097473bd90774d268', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.726900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.727245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.727602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.728372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:35.728716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.730406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.731356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.732571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.733132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.733644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] PATH_INFO: `/v3/users/e959a80eb9c84665862a5d93e58df05c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.735553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e959a80eb9c84665862a5d93e58df05c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.738476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.752038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 975/1945] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:25:35.754713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.757000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.775955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d43f-baa4-4acd-8e75-d78ecba04853 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:35.784322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8fd94c7188f4c068102c986f952ef6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6693809e91a497097473bd90774d268', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:35.785202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:35.785569 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:35.785919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:35.787452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:35.790861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:35.834260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e959a80eb9c84665862a5d93e58df05c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.834516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 976/1946] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:35 2022] POST /identity/v3/projects => generated 353 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:35.837871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e959a80eb9c84665862a5d93e58df05c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.839099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:35.843040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:35.849566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e959a80eb9c84665862a5d93e58df05c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.877897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e959a80eb9c84665862a5d93e58df05c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.898258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e959a80eb9c84665862a5d93e58df05c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:35.923366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 971/1947] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:35 2022] DELETE /identity/v3/users/e959a80eb9c84665862a5d93e58df05c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:35.974161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012edb0-c53f-44d5-bffb-a11c3bc7988f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:36.036841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8fd94c7188f4c068102c986f952ef6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6693809e91a497097473bd90774d268', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:36.037931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.038318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.038709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.039633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:36.040118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:36.041797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:36.060288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 977/1948] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:36.096547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:36.108027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ca0d0-b25b-44c6-a5eb-ba9e0c5f4009 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:36.114611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8fd94c7188f4c068102c986f952ef6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6693809e91a497097473bd90774d268', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:36.115791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.116360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.116766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] PATH_INFO: `/v3/projects/1b36479bbfc04a3e9294d24e8ac63217` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.118382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1b36479bbfc04a3e9294d24e8ac63217)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:36.120432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:36.159191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:36.190412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b36479bbfc04a3e9294d24e8ac63217', 'request_id': 'req-74180884-fe1c-4abf-ae02-ec525970ed78'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.198803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b36479bbfc04a3e9294d24e8ac63217', 'request_id': 'req-74180884-fe1c-4abf-ae02-ec525970ed78'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.206567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b36479bbfc04a3e9294d24e8ac63217', 'request_id': 'req-74180884-fe1c-4abf-ae02-ec525970ed78'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.218102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] The token cache is being invalidate because project 1b36479bbfc04a3e9294d24e8ac63217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:36.218395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.221590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.225403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 972/1949] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:36 2022] DELETE /identity/v3/projects/1b36479bbfc04a3e9294d24e8ac63217 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:36.329704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:36.332146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.483842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebc58b-1365-452a-8e74-16ad8663060f tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:36.493580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:36.566280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:36.567244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.567663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.568013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] PATH_INFO: `/v3/projects/3a006ff187b9412bb5e46400e63c63e0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.569294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a006ff187b9412bb5e46400e63c63e0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:36.571581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:36.592410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:36.595205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.616793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74180884-fe1c-4abf-ae02-ec525970ed78 tempest-VolumeQuotaClassesTest-1034548256 tempest-VolumeQuotaClassesTest-1034548256-project-admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:36.623728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:36.624694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.624993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.625281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] PATH_INFO: `/v3/users/36fec93a443d4be890257f144bdaa8d9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.627803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fec93a443d4be890257f144bdaa8d9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:36.629970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:36.639799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a006ff187b9412bb5e46400e63c63e0', 'request_id': 'req-24207302-eca6-40b7-ae75-0903b08deb1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.649914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a006ff187b9412bb5e46400e63c63e0', 'request_id': 'req-24207302-eca6-40b7-ae75-0903b08deb1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.658801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a006ff187b9412bb5e46400e63c63e0', 'request_id': 'req-24207302-eca6-40b7-ae75-0903b08deb1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.676404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] The token cache is being invalidate because project 3a006ff187b9412bb5e46400e63c63e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:36.677871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.678924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-24207302-eca6-40b7-ae75-0903b08deb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.684712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 978/1950] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:36 2022] DELETE /identity/v3/projects/3a006ff187b9412bb5e46400e63c63e0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:36.687719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defe70-460e-4ba1-a6ee-442a9f24fc88 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.688082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defe70-460e-4ba1-a6ee-442a9f24fc88 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.690150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16defe70-460e-4ba1-a6ee-442a9f24fc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.698138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fec93a443d4be890257f144bdaa8d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.699494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fec93a443d4be890257f144bdaa8d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.713808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fec93a443d4be890257f144bdaa8d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.716806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-16defe70-460e-4ba1-a6ee-442a9f24fc88 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:36.718800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16defe70-460e-4ba1-a6ee-442a9f24fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.741351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fec93a443d4be890257f144bdaa8d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.757960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc839fdf-5323-46ec-89c6-99b4741ff700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fec93a443d4be890257f144bdaa8d9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:36.765771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 973/1951] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:36 2022] DELETE /identity/v3/users/36fec93a443d4be890257f144bdaa8d9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:36.772274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.773290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.773583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.802075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:36.803435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.818986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 979/1952] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:36.827635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.828249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.828727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.850463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:36.852246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.907855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 974/1953] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:36.907999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:36.920786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:36.922421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:36.952098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 980/1954] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:36.965602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:36.979428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e45b35-40d7-4707-966f-67b0cb32b9ed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:36.985981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:36.986811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:36.987102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:36.987416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:36.988225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:36.988539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:36.990267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.001504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:37.029686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 981/1955] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:37.030823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.048699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225aef9-cdb4-40d6-b609-ec7e2ca64551 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.049383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d9ef6-501a-4292-bba9-a7a9a9ef2ab6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.053672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.054579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.055055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.055538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.058163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.058841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.059124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.060273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.061117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.063253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.076469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.087017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.099873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 975/1956] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:37.103328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 982/1957] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:37 2022] POST /identity/v3/projects => generated 361 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:37.107712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.120306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdbf97-a7d2-4ca5-afb1-b7624bd2d8b6 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.207941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.208846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.209233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.209616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.211360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.216047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.249506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 976/1958] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:37 2022] POST /identity/v3/users => generated 281 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:37.254000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.270648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5c29-ff84-40ca-89af-e33bb3629118 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.275950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.276263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.276556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.276849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.277623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.279259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.295622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 983/1959] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:37.296319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.309536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea68577-1919-4ab3-bc86-c77bfb1240de admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.315450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.316209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.316501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.316820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.318151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.319984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c790dc-4ec3-4f39-9de0-7fc639f9ee95 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.333297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 977/1960] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:37.337506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.356894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81679-01db-4da1-86a9-24e0e5e9ba33 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.362401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.363428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.363782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.364155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] PATH_INFO: `/v3/projects/dfca0823c41b4e6c918659ec891bfe96/users/0e054f247483456fa9544095dacf89cf/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.385889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfca0823c41b4e6c918659ec891bfe96, user_id=0e054f247483456fa9544095dacf89cf, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.388904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.405916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 984/1961] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:37 2022] PUT /identity/v3/projects/dfca0823c41b4e6c918659ec891bfe96/users/0e054f247483456fa9544095dacf89cf/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:37.418732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.419184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.419612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.456888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] MFA Rules not processed for user `0e054f247483456fa9544095dacf89cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:37.458235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.464295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.466133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.569307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 978/1962] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:37.604025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.605577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.606491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4811d5-cbf9-41cd-bd3d-65805aba031d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.613720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.614666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.614959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.615275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] PATH_INFO: `/v3/projects/87d903e3ff5446179523a1598cdb0342` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.616641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d903e3ff5446179523a1598cdb0342)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.619203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.653741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:37.680209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184198-daf2-4055-bcb9-34a26e789d5e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.706671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d903e3ff5446179523a1598cdb0342', 'request_id': 'req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.727303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d903e3ff5446179523a1598cdb0342', 'request_id': 'req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.727303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d903e3ff5446179523a1598cdb0342', 'request_id': 'req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.741624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:37.746216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] The token cache is being invalidate because project 87d903e3ff5446179523a1598cdb0342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:37.746728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.747939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.749851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 985/1963] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:37 2022] DELETE /identity/v3/projects/87d903e3ff5446179523a1598cdb0342 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:37.753833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.755564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.755737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.756405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.756703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.756972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.758897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.818468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.819957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.820499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cacebe-1b81-48ed-b7b6-f1d4b3b84d04 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.825271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.826153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.826440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.826720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] PATH_INFO: `/v3/users/a8fd94c7188f4c068102c986f952ef6c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.827937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8fd94c7188f4c068102c986f952ef6c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.828748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 979/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:37.830559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.842836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:37.844284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:37.858527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24e57d-968f-478a-8d7c-c9c91e26254d service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:37.863802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:37.864869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.865298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.865726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] PATH_INFO: `/v3/users/0e054f247483456fa9544095dacf89cf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.866956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e054f247483456fa9544095dacf89cf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:37.868903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:37.878824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8fd94c7188f4c068102c986f952ef6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.892215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8fd94c7188f4c068102c986f952ef6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.900445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8fd94c7188f4c068102c986f952ef6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.908959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8fd94c7188f4c068102c986f952ef6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.909624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1db6454-49d2-47f0-9df4-1d591298a29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8fd94c7188f4c068102c986f952ef6c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.915834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 986/1965] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:37 2022] DELETE /identity/v3/users/a8fd94c7188f4c068102c986f952ef6c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:37.917512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e054f247483456fa9544095dacf89cf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.918299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e054f247483456fa9544095dacf89cf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.925431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e054f247483456fa9544095dacf89cf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.938865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e054f247483456fa9544095dacf89cf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.946395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e054f247483456fa9544095dacf89cf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:37.952561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 980/1966] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:37 2022] DELETE /identity/v3/users/0e054f247483456fa9544095dacf89cf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:37.957807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:37.958102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:37.958386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:37.980290 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:37.981748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.083584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 987/1967] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:38.096849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.099252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.134511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:38.189610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ec5f-9345-4c04-84b5-e4cd8012048d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.193654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.194366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.194652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.194949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.206604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.208208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579a7ba-0fe7-42c1-8522-d72ebdbbf8f0 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.219616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 981/1968] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:38.227730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.229182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.256565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed9027-eb1d-47b3-81de-870144aa2ea3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.263705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.264243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.264533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.265141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] PATH_INFO: `/v3/users/0deceba75df8425380687501594c6e0d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.266450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0deceba75df8425380687501594c6e0d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.269993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.308644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:38.376553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0deceba75df8425380687501594c6e0d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.393468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0deceba75df8425380687501594c6e0d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.402701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0deceba75df8425380687501594c6e0d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.424551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0deceba75df8425380687501594c6e0d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.425526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0deceba75df8425380687501594c6e0d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.455400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 988/1969] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:38 2022] DELETE /identity/v3/users/0deceba75df8425380687501594c6e0d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:38.457136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.457436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.457740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.485961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:38.487480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.546404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.549325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.581125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 982/1970] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:38.596931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.598414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.655757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34aeb8e-1d6e-4882-854f-4e4c4e759286 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.662396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.663318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.663604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.663883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] PATH_INFO: `/v3/projects/b6693809e91a497097473bd90774d268` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.665116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6693809e91a497097473bd90774d268)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.667065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.676851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c016a4-cd2a-42d5-8c07-39dc32ca70df None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.682230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.683485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.683485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.683823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.696447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.698052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.717857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 983/1971] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:38.720544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.727337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.743625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6693809e91a497097473bd90774d268', 'request_id': 'req-89bf570c-3445-4fb8-9961-a2168aa3fb72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.747290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57766a01-a171-47d0-99de-f463e4f2b267 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.754352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6693809e91a497097473bd90774d268', 'request_id': 'req-89bf570c-3445-4fb8-9961-a2168aa3fb72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.765244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6693809e91a497097473bd90774d268', 'request_id': 'req-89bf570c-3445-4fb8-9961-a2168aa3fb72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.788184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] The token cache is being invalidate because project b6693809e91a497097473bd90774d268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:38.788598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.789330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89bf570c-3445-4fb8-9961-a2168aa3fb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.791663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 989/1972] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:38 2022] DELETE /identity/v3/projects/b6693809e91a497097473bd90774d268 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:38.797059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422a8b-97bd-4afc-80c7-27f06b91c24c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.797426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422a8b-97bd-4afc-80c7-27f06b91c24c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.798281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422a8b-97bd-4afc-80c7-27f06b91c24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.826424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4422a8b-97bd-4afc-80c7-27f06b91c24c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:38.827849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4422a8b-97bd-4afc-80c7-27f06b91c24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.830425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.831345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.831637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.831988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] PATH_INFO: `/v3/projects/dfca0823c41b4e6c918659ec891bfe96` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.833114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfca0823c41b4e6c918659ec891bfe96)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.835038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.836609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 990/1973] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:38.843984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:38.845777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.846245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.846478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.866225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:38.867795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.877124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 991/1974] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:38.881265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.890892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542a7d3-4da4-4ac1-ad69-b1b256e3bb17 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:38.897308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:38.907482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfca0823c41b4e6c918659ec891bfe96', 'request_id': 'req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.917289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfca0823c41b4e6c918659ec891bfe96', 'request_id': 'req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.928636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfca0823c41b4e6c918659ec891bfe96', 'request_id': 'req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.944483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] The token cache is being invalidate because project dfca0823c41b4e6c918659ec891bfe96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:38.945041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.945459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9dbf96a1-7d37-4b06-94d5-eb1caf3347f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:38.947285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 984/1975] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:38 2022] DELETE /identity/v3/projects/dfca0823c41b4e6c918659ec891bfe96 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:38.951692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690656-81d2-4288-babd-f012bcd1549c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.952182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690656-81d2-4288-babd-f012bcd1549c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.952308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8690656-81d2-4288-babd-f012bcd1549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.959089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:38.960073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:38.960448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:38.960805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:38.961851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:38.962236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:38.964123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:38.979274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 992/1976] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:25:38.984406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e8690656-81d2-4288-babd-f012bcd1549c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:38.985705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:38.986309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8690656-81d2-4288-babd-f012bcd1549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:38.988571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.003777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 985/1977] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:39.006922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.007441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.007879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.008887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e043bc-b555-4e0f-bb43-8ebad440b0fe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.014578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.015506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.015864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.016238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.017216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.019259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.030076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:39.032979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.044644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 986/1978] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:39.049429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.060028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 993/1979] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:38 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:39.061006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.062324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1381cb-7cf7-4316-b89c-cd4b5a8d48f2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.133055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.133738 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.134044 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.134297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.135049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:39.135390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.137363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.145708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2870f9-67ee-4f7f-9d52-c2f4bd654634 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.154317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 987/1980] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:25:39.154317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.155815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.156867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.158077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.158367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.158649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.160680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.162686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.175346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094cff6-00a2-4f85-b000-4dfcbaa95e65 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.186174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.186782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 994/1981] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:39.187659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.187928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.188033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.188265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] PATH_INFO: `/v3/projects/788f99e1f77946fe9f0771ffff9e934c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.190626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=788f99e1f77946fe9f0771ffff9e934c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.192617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.200227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc179cd-6662-41e7-9e30-1c6b6ddadf9b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.206165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.207084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.207192 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.207424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.208118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.210297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.247472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 995/1982] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:39.248701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.251076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788f99e1f77946fe9f0771ffff9e934c', 'request_id': 'req-b7c66299-4ec6-4696-8269-5a30e919f11d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.265063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788f99e1f77946fe9f0771ffff9e934c', 'request_id': 'req-b7c66299-4ec6-4696-8269-5a30e919f11d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.274052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788f99e1f77946fe9f0771ffff9e934c', 'request_id': 'req-b7c66299-4ec6-4696-8269-5a30e919f11d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.283312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:39.288124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] The token cache is being invalidate because project 788f99e1f77946fe9f0771ffff9e934c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:39.288427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.289277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.291969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 988/1983] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:39 2022] DELETE /identity/v3/projects/788f99e1f77946fe9f0771ffff9e934c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:39.296647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.298396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.314473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:39.331246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5b06b-716d-4332-8901-41a7b3c376cc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.336887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.337512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.337798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.338087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.338800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.340434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.354639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 996/1984] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:39.354639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.356464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.375564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c66299-4ec6-4696-8269-5a30e919f11d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.378687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c5732-841f-4793-a178-14d0aae341fa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.380674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.381357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.381653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.383457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.383457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.386557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.387671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.387976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.388243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] PATH_INFO: `/v3/users/002a10f98a1b478abca92599b5a14893` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.389616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002a10f98a1b478abca92599b5a14893)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.389888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.392582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.420289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 989/1985] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:39.421123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.422946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.438307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0a19-14c3-4f30-9c2e-50197aa702c8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.454930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.456933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.457205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.457491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.459719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.462542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.463278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002a10f98a1b478abca92599b5a14893'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.480578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002a10f98a1b478abca92599b5a14893'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.494299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002a10f98a1b478abca92599b5a14893'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.502919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 990/1986] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:39.503805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.508205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002a10f98a1b478abca92599b5a14893'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.517163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002a10f98a1b478abca92599b5a14893'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:39.528995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a8597-2977-47cb-96f1-f260291a28d2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.535642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.536408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.536694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.539841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 997/1987] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:39 2022] DELETE /identity/v3/users/002a10f98a1b478abca92599b5a14893 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:39.539983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.540073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.540148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.540455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.562799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.610211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 991/1988] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:39.614450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.617757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.664339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200dbbec-1a7d-4f18-9cef-e3aa2a40ef8d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.674674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae42b-15a1-41a2-8a58-9419e40a2d68 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.682089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.683109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.683437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.684206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] PATH_INFO: `/v3/projects/dd1ae2f16f2d42eea3d9797a8fc7fae7/users/b3ccc67ded4144559aa3f84fd4a598f7/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.694495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.695524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.695922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.696608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.697403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.699320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.705803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1ae2f16f2d42eea3d9797a8fc7fae7, user_id=b3ccc67ded4144559aa3f84fd4a598f7, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.713272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580b6b7-47df-41fe-9702-98638c8bc752 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.726828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 992/1989] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:39.732041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.735806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 998/1990] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:39 2022] PUT /identity/v3/projects/dd1ae2f16f2d42eea3d9797a8fc7fae7/users/b3ccc67ded4144559aa3f84fd4a598f7/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:39.748652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.749235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.749606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.761848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2c54-2857-47fb-8917-d7e16364925d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.794085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] MFA Rules not processed for user `b3ccc67ded4144559aa3f84fd4a598f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:39.797270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:39.856376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:39.912439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 999/1991] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:39.917585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:39.920729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.921697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.921987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.922275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] PATH_INFO: `/v3/projects/3c78c1137c8f46de92ea1cd9840668f4/users/516b207adb3444da982414fa3f1feebd/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.931778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda72cb-80c3-4eb4-ae61-addc41560faa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:39.937479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:39.938983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:39.939369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:39.940277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:39.940661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.942976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:39.957707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c78c1137c8f46de92ea1cd9840668f4, user_id=516b207adb3444da982414fa3f1feebd, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:39.960376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9485188-0eee-4509-bdb2-b4f69f609430 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.003895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1000/1993] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:39 2022] POST /identity/v3/projects => generated 387 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:40.004673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.007256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 993/1993] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:39 2022] PUT /identity/v3/projects/3c78c1137c8f46de92ea1cd9840668f4/users/516b207adb3444da982414fa3f1feebd/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:40.021752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.022039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.022327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.074618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] MFA Rules not processed for user `516b207adb3444da982414fa3f1feebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:40.075897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.118004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f617e-3b86-4621-8925-e0197e575424 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.120483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:40.126566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.127437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.135040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.135465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.136384 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.139840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.178954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1001/1994] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:40.180357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.212499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72a00-fca1-4e82-bcb4-a1ff95ac6ea6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.221535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.223310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.223793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.224256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.225363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.227760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 994/1995] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:40.228566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.228988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.244138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.258072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1002/1996] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:40.259320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.261278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.270282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d086-cb0c-4f50-b088-b49b310fb29e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.281186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.282748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.283100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.283849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] PATH_INFO: `/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.285609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b9e48c44c545a6b89e5bd66f407a1d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.288796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.357933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c925-05c4-4851-aa08-6e0df9d7da50 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.364907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.365885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.366244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.366523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.380890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.383802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.391883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b9e48c44c545a6b89e5bd66f407a1d', 'request_id': 'req-4a1fa143-a645-4faa-907a-782a02e674cb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:40.396604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1003/1997] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:40.397905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.409721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b9e48c44c545a6b89e5bd66f407a1d', 'request_id': 'req-4a1fa143-a645-4faa-907a-782a02e674cb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:40.415479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec790595-1e9e-46cd-b1c6-b3052da543ab service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.431078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b9e48c44c545a6b89e5bd66f407a1d', 'request_id': 'req-4a1fa143-a645-4faa-907a-782a02e674cb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:40.444957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] The token cache is being invalidate because project 98b9e48c44c545a6b89e5bd66f407a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:40.445373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:40.446196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4a1fa143-a645-4faa-907a-782a02e674cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:40.448083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 995/1998] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:40 2022] DELETE /identity/v3/projects/98b9e48c44c545a6b89e5bd66f407a1d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:40.454952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94ecd-e22e-4aa8-a55e-d2784fd9bc40 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.455409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94ecd-e22e-4aa8-a55e-d2784fd9bc40 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.455796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94ecd-e22e-4aa8-a55e-d2784fd9bc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.464490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:40.481316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e8f94ecd-e22e-4aa8-a55e-d2784fd9bc40 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:40.482499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8f94ecd-e22e-4aa8-a55e-d2784fd9bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.509967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.510652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.510945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.511244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.512280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.514726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.528968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1004/1999] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:40.534181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.537375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.559239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 996/2000] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:40.559383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dc2d6-69c8-4cc5-9a55-903028917507 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.568767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.569142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.569555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.572523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.573452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.573894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.574169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] PATH_INFO: `/v3/projects/269d2ba7b4b54ae7aa6446c1d056a6a9/users/aa3fbd14b85b445999be79cc98484ec1/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.595124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:40.596800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.597966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=269d2ba7b4b54ae7aa6446c1d056a6a9, user_id=aa3fbd14b85b445999be79cc98484ec1, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.601935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf075cb-d95e-4402-b3c2-0bff18c4d0ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.617978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 997/2001] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:40.618700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.630451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716d86-65df-42d2-91e6-e736c84c55e8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.644273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1005/2002] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:40 2022] PUT /identity/v3/projects/269d2ba7b4b54ae7aa6446c1d056a6a9/users/aa3fbd14b85b445999be79cc98484ec1/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:40.646150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.646692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.646809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.680898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] MFA Rules not processed for user `aa3fbd14b85b445999be79cc98484ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:40.682449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:40.691565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:40.715425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.716556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.716853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.717141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.717896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:40.718212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.721930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.752226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 998/2003] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:40.765910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.784665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4d97d-fb06-4d42-b89a-540924dd43cd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.800537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.801228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.801520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.801807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.802520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.804670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.851749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1006/2004] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:40.856804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.865469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 999/2005] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:40.871723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:40.888039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f570fb-95f9-4832-a6d7-db6ade44994c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.989648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1188e07-b678-4a47-85c7-6f69e15c2ace admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:40.994863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.995727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.996271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.996638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.997390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:40.997510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:40.998088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:40.998368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:40.998647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:40.999636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:40.999746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.002854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.035527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1007/2006] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:41.036577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.042342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.042853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1000/2007] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:40 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:41.044058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.057756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97c41c-81ed-4dfd-9c2a-e13ba033510b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.070181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:41.141010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd502d4f-5166-4edd-afb7-2809efccf2c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.149295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.150421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.153997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.154206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] PATH_INFO: `/v3/users/14de3d1e5a1841389bc8ccabde1e73d1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.157935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14de3d1e5a1841389bc8ccabde1e73d1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.160061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.160754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.161456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.161742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.162020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.162722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.168827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.197210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1001/2008] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:41 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:41.198124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.209693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86570c46-61fa-47c7-abb6-d1c177403b22 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.218633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.220007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.220308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.220590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.222029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.223056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14de3d1e5a1841389bc8ccabde1e73d1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:41.224705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.248245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1002/2009] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:41.252260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.254166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14de3d1e5a1841389bc8ccabde1e73d1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:41.264578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14de3d1e5a1841389bc8ccabde1e73d1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:41.269942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e41287-4c90-4f89-a7c5-21759c5359d1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.274190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14de3d1e5a1841389bc8ccabde1e73d1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:41.275262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14de3d1e5a1841389bc8ccabde1e73d1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:41.283332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.284025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.284315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.284562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1008/2010] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:41 2022] DELETE /identity/v3/users/14de3d1e5a1841389bc8ccabde1e73d1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:41.285690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.287392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.287629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.289075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.290811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2cb2d-09cc-4072-b937-75d7eb26856e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.306492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1003/2011] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:41.306492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.306492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.306492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.340749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:41.342631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.354326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:41.390243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae0236-2cd2-43fd-945e-9389941ad892 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.395562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.396791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.397153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.397507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.398503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.400447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.413984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1009/2012] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:41.414805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.416684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.436658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756138c4-02f1-4b2c-9555-6b1b7b45bf4e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.437423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1004/2013] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:41.438618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.447504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.448422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.448707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.448983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] PATH_INFO: `/v3/projects/2188796a964a4e70b06c3a3e04af2767/users/8ec8127acd53414d8b428dab88ab4c7f/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.449408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e5f4f-ad5e-4e4d-aa75-18bb24ea21c6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.455115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.456055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.456340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.456620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] PATH_INFO: `/v3/projects/ca9476b788ba4624ba8053c5a0a2324b/users/5d0f89a7ad674aa5931a276ea82b97f1/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.468152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2188796a964a4e70b06c3a3e04af2767, user_id=8ec8127acd53414d8b428dab88ab4c7f, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.472395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.473132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9476b788ba4624ba8053c5a0a2324b, user_id=5d0f89a7ad674aa5931a276ea82b97f1, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.475993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.488899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1010/2014] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:41 2022] PUT /identity/v3/projects/2188796a964a4e70b06c3a3e04af2767/users/8ec8127acd53414d8b428dab88ab4c7f/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:41.489020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.490658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.494310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1005/2015] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:41 2022] PUT /identity/v3/projects/ca9476b788ba4624ba8053c5a0a2324b/users/5d0f89a7ad674aa5931a276ea82b97f1/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:41.495131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.506184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:41.563653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9aeb2-85cc-4398-9451-5fd029566de7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.564125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa221-0c58-45dc-b381-6da8fe760ccf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.568884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.569561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.569850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.570136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.570880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.572521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da9ac9-eb6f-460d-955b-f47c01d914ad admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.585922 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1006/2016] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:41.587388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.587687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.588085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.616629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] MFA Rules not processed for user `5d0f89a7ad674aa5931a276ea82b97f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:41.617984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.619870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.620717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.621003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.621283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.632554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.633998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.646495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1011/2017] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:41.653027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.670794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f744e16-f80d-43ef-9a64-24e8c4de6ccb service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.676102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.677707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.677707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.678296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] PATH_INFO: `/v3/projects/2188796a964a4e70b06c3a3e04af2767/users/8ec8127acd53414d8b428dab88ab4c7f/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.680722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2188796a964a4e70b06c3a3e04af2767, user_id=8ec8127acd53414d8b428dab88ab4c7f, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.685146 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360e52b-0eb6-41c4-8c17-fbd1976d804c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.697682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1007/2018] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:41.698686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:41.699981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1012/2019] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:41 2022] PUT /identity/v3/projects/2188796a964a4e70b06c3a3e04af2767/users/8ec8127acd53414d8b428dab88ab4c7f/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:41.700293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.710781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.711167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.711611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.721114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:41.748479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] MFA Rules not processed for user `8ec8127acd53414d8b428dab88ab4c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:41.753614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:41.780896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a71813-6a30-41a6-90e4-1460709f7f36 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:41.841827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:41.842515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:41.842809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:41.843088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:41.924959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1013/2020] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:41.965382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:41.967158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:41.980134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1008/2021] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:41.999556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.001458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.009490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.025686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99838f3c-cae9-4377-9002-0b2d183361a5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.033253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.033891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.034182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.034493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.049882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.050573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.063076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1014/2022] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:42.063993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.065394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.112784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:42.139375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5c862-6f57-4045-b924-501bc4c66587 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.154773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.156279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.157501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.157790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] PATH_INFO: `/v3/users/5d0f89a7ad674aa5931a276ea82b97f1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.159011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0f89a7ad674aa5931a276ea82b97f1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.163556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.171077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae2e19-b7f8-4b7b-bc73-764cfe1b98c4 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.177283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.178289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.178644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.179046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] PATH_INFO: `/v3/projects/d1251bd6ebe64bf4b1c0812da0400a05` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.180348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1251bd6ebe64bf4b1c0812da0400a05)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.182328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.208417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:42.246859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1251bd6ebe64bf4b1c0812da0400a05', 'request_id': 'req-d909f5f6-0067-485b-9d68-eccf6b933ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.255806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0f89a7ad674aa5931a276ea82b97f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.256450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1251bd6ebe64bf4b1c0812da0400a05', 'request_id': 'req-d909f5f6-0067-485b-9d68-eccf6b933ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.257510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0f89a7ad674aa5931a276ea82b97f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.265255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1251bd6ebe64bf4b1c0812da0400a05', 'request_id': 'req-d909f5f6-0067-485b-9d68-eccf6b933ac7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.275034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0f89a7ad674aa5931a276ea82b97f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.301396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0f89a7ad674aa5931a276ea82b97f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.311418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbcc75dc-a95b-4a77-826e-2fff612e9502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0f89a7ad674aa5931a276ea82b97f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.320831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1009/2023] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:42 2022] DELETE /identity/v3/users/5d0f89a7ad674aa5931a276ea82b97f1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:42.323799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] The token cache is being invalidate because project d1251bd6ebe64bf4b1c0812da0400a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:42.324119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.325604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.326138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.326261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d909f5f6-0067-485b-9d68-eccf6b933ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:42.326342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.339084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1015/2024] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:42 2022] DELETE /identity/v3/projects/d1251bd6ebe64bf4b1c0812da0400a05 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:42.348392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39b62-12dd-47ab-8b65-46f5c941efa8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.351490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39b62-12dd-47ab-8b65-46f5c941efa8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.351929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a39b62-12dd-47ab-8b65-46f5c941efa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.353157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:42.354456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.406485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-24a39b62-12dd-47ab-8b65-46f5c941efa8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:42.410769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24a39b62-12dd-47ab-8b65-46f5c941efa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.501250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1010/2025] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:42.519257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.521788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.582229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1016/2026] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:42.599473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.599934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.600626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.633944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4d6ae-0f62-4f38-a40c-7c4f6d5b605e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.638892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.640317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.640666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.642058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.648542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:42.649799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:42.666307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.668468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.677141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1017/2027] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:42.683325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.689645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1011/2028] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:42.706944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fca6e-f604-4403-a1a8-65950a7a8cef None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.721294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.722076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.722618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.722919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.723904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:42.724250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.726818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.742490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1018/2029] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:42.746675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.769360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8847-9b0a-457a-81d4-4b75c5837452 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.775896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.776555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.776844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.777181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.777904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.780400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:42.823437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1012/2030] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:42 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:42.824225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:42.838251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22163-c055-4037-b11f-9a81ec80ba26 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:42.902008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:42.952028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:42.952777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:42.953154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:42.953443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:42.957033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:42.959658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.008356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1019/2031] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:42 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:43.012414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.045960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ae48a-e559-4f0f-92bf-3058a3cfc59a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.053767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.058173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.058472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.058754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.059536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.061169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.090412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1013/2032] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:43.099435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.116668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341c9aa-b882-400d-bce6-06d3e80466fd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.122482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.126459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.129254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.130468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.131035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.131266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.132318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.139371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.160379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:43.170943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1020/2033] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:43.176371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.213445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5667fa4-910f-4ccc-a336-1562861b8408 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.217647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411e4ac-7ed1-455f-8205-1e3cbdf05430 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.222502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.223479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.224621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.225288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.228719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.231006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.232546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.232824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.233104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] PATH_INFO: `/v3/projects/a75a2d018dff40bfb9b5bd72a678826c/users/74581e9c5b27401686b40b2312a6f2fb/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.255003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75a2d018dff40bfb9b5bd72a678826c, user_id=74581e9c5b27401686b40b2312a6f2fb, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.258034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b0bbe-a4c6-4add-81b3-12154e376539 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.288880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1021/2034] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:43 2022] PUT /identity/v3/projects/a75a2d018dff40bfb9b5bd72a678826c/users/74581e9c5b27401686b40b2312a6f2fb/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:43.294632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.294964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.295292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.327443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:43.339622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.348730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.350720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.364373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1014/2035] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:43.387342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.389107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.409752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc441b-0cbf-491c-962e-7d6bbf4a99a9 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.447259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1022/2036] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:43.457113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.458512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.494032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.494996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.495445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.495858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] PATH_INFO: `/v3/projects/ca9476b788ba4624ba8053c5a0a2324b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.497405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9476b788ba4624ba8053c5a0a2324b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.499556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.546755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e8d3c-395b-453d-b504-ac38cffeea17 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.568608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9476b788ba4624ba8053c5a0a2324b', 'request_id': 'req-e6dfb112-9d2d-4161-9248-08d85cac5ab8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:43.582272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9476b788ba4624ba8053c5a0a2324b', 'request_id': 'req-e6dfb112-9d2d-4161-9248-08d85cac5ab8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:43.601439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9476b788ba4624ba8053c5a0a2324b', 'request_id': 'req-e6dfb112-9d2d-4161-9248-08d85cac5ab8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:43.611510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:43.620197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] The token cache is being invalidate because project ca9476b788ba4624ba8053c5a0a2324b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:43.620501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:43.624136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6dfb112-9d2d-4161-9248-08d85cac5ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:43.629723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1015/2037] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:43 2022] DELETE /identity/v3/projects/ca9476b788ba4624ba8053c5a0a2324b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:43.635561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b619-3228-4b36-ac10-a1478b17db86 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.635856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b619-3228-4b36-ac10-a1478b17db86 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.636143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885b619-3228-4b36-ac10-a1478b17db86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.652481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:43.679615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.680306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.680601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.680741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9885b619-3228-4b36-ac10-a1478b17db86 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:43.681066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.681908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9885b619-3228-4b36-ac10-a1478b17db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.690552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.798411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1016/2038] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:43.802440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.803088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.803088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.834758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:43.836256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:43.850752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1017/2039] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:43.851703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.861046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.862603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.862942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b82f7-0954-4fba-9846-cf2c29454049 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.867934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.868875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.869077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.869364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.870117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:43.870429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.872263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:43.878060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1023/2040] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:43.890668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1018/2041] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:43.891969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:43.908246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74acc74-d686-432c-9dba-6556b1250e5e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:43.914503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:43.916141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:43.916942 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:43.917513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:43.919554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:43.924335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.003313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1024/2042] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:43 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:44.005773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.024026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f746bac-f52b-4828-8ed3-a4d8a515b863 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.123137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.124257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.124543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.124826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.126204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.128206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.156018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1019/2043] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:44 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:44.158201 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.169418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5f17-60dd-4530-8621-4b2452de9ed6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.175049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.175792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.176083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.176363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.177108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.178708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.189860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1025/2044] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:44.193851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.204549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c346640-3a3c-4391-a076-4e852e69fa48 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.209300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.209978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.210270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.210556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.211322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.213425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdde16e-300c-4a37-a714-f0700059feb5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.224796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1020/2045] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:44.229119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.252700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b858cf-215e-4972-a1ef-b2b03354aa51 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.258251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.259155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.259499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.259782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] PATH_INFO: `/v3/projects/56c6b45912e64ad691232fc05df8c123/users/242a289932b34e06ac763d96c6315037/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.276165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c6b45912e64ad691232fc05df8c123, user_id=242a289932b34e06ac763d96c6315037, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.278835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.309014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1026/2046] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:44 2022] PUT /identity/v3/projects/56c6b45912e64ad691232fc05df8c123/users/242a289932b34e06ac763d96c6315037/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:44.321410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.321705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.322427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.349339 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] MFA Rules not processed for user `242a289932b34e06ac763d96c6315037`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:44.350723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.370870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.373727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.444621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:44.499604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1021/2047] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:44.514908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.517785 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.525468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8bb69-8335-4756-a494-53be49b1e2be admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.533584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.534527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.534824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.535172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] PATH_INFO: `/v3/users/516b207adb3444da982414fa3f1feebd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.536918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516b207adb3444da982414fa3f1feebd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.540028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.605404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bbf1e-3feb-44f0-baad-925564bb99ba None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.609293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.610363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.610670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.610955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.611915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:44.621685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516b207adb3444da982414fa3f1feebd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:44.621929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.623395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a621c35-665e-4140-a27b-fa9bb58ac180 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.633671 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1022/2048] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:44.636784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516b207adb3444da982414fa3f1feebd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:44.645813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516b207adb3444da982414fa3f1feebd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:44.653973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516b207adb3444da982414fa3f1feebd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:44.654690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516b207adb3444da982414fa3f1feebd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:44.662708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1027/2049] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:44 2022] DELETE /identity/v3/users/516b207adb3444da982414fa3f1feebd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:44.667629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9eb7-e653-478b-9fa5-f587c53646d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.668158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9eb7-e653-478b-9fa5-f587c53646d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.668568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9eb7-e653-478b-9fa5-f587c53646d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.689681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-871b9eb7-e653-478b-9fa5-f587c53646d6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:44.690976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-871b9eb7-e653-478b-9fa5-f587c53646d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.762613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1023/2050] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:44.774828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:44.776541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.812744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.813883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.814421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.854536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] MFA Rules not processed for user `74581e9c5b27401686b40b2312a6f2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:44.856305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:44.882840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cacc3-a3b8-46db-8efb-84c7dba94458 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:44.890716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:44.891947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:44.892464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:44.892934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:44.914301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:44.917820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:44.953144 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1028/2051] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:45.052904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1024/2052] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:45.317846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:45.320114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:45.404272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4e2c1-d4aa-4da4-ad37-8bf2506fbc0f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:45.409912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:45.410593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:45.410882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:45.411486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:45.422043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:45.424527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a354409-c459-42fa-bebd-39d2e6464926 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:45.437885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1029/2053] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:45.746817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:45.746817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:45.767966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf630cab-e267-482a-8893-636d9ae94ed9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:45.782236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:45.783134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:45.783442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:45.783719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] PATH_INFO: `/v3/projects/3c78c1137c8f46de92ea1cd9840668f4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:45.785902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c78c1137c8f46de92ea1cd9840668f4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:45.788352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:45.824542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:45.862757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c78c1137c8f46de92ea1cd9840668f4', 'request_id': 'req-c0e2485c-ef69-4613-83ed-92187f139a45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:45.874707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c78c1137c8f46de92ea1cd9840668f4', 'request_id': 'req-c0e2485c-ef69-4613-83ed-92187f139a45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:45.882334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c78c1137c8f46de92ea1cd9840668f4', 'request_id': 'req-c0e2485c-ef69-4613-83ed-92187f139a45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:45.894803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] The token cache is being invalidate because project 3c78c1137c8f46de92ea1cd9840668f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:45.895182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:45.897252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c0e2485c-ef69-4613-83ed-92187f139a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:45.899153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1025/2054] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:45 2022] DELETE /identity/v3/projects/3c78c1137c8f46de92ea1cd9840668f4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:45.903959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:45.904198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:45.904515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:45.923987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:45.925490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.004323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1030/2055] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:46.012379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.012686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.013012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.032367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:46.033631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.043700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1026/2056] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:46.049095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.057322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:46.061640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9612d1-7249-419c-b7e3-3b74591e29f5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.069372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.070085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.070414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.070715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.072155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:46.072653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.075398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.092244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1031/2057] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:46.093688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.103299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c5247-d932-4467-9077-a4f229f93a03 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.108735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.108735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.109067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.109367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.110567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.112528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.139180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1027/2058] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:46.143354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.153371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcb678-b016-42df-92bc-fe1cc40d9747 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.211424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.212109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.212396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.212673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.213364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.215303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8edde9-83a5-4b5d-a273-29c82a3bfb37 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.260273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1032/2059] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:46.265271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.277399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602c919-757b-4505-8b72-838c58ed983f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.282972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.283700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.283984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.284265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.285001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.286589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.298507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2432a-3829-4da2-b943-baf0025bfea3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.298846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2432a-3829-4da2-b943-baf0025bfea3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.299595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2432a-3829-4da2-b943-baf0025bfea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.307267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1028/2060] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:46.309547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.318447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dbc2432a-3829-4da2-b943-baf0025bfea3 None None] MFA Rules not processed for user `242a289932b34e06ac763d96c6315037`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:46.319676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dbc2432a-3829-4da2-b943-baf0025bfea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.331332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65b662-d4ce-4d55-834e-bd9c8ebc1b72 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.336758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.337621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.338207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.338491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.339254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.341286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.354253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1029/2061] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:25:46.358194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.370628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ed8ee-7a12-4b71-b9fb-2053093bdd83 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.376395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.377293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.377586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.377873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] PATH_INFO: `/v3/projects/ee14fb4cfead402ab57d78792f303b48/users/857d941f699143ecbf25d842fb1f9ff8/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.393557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee14fb4cfead402ab57d78792f303b48, user_id=857d941f699143ecbf25d842fb1f9ff8, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.394454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1033/2062] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:46.396397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.419129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1030/2063] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:46 2022] PUT /identity/v3/projects/ee14fb4cfead402ab57d78792f303b48/users/857d941f699143ecbf25d842fb1f9ff8/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:46.420833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed4c0-3e9f-43e7-a059-effd68b0593a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.421340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed4c0-3e9f-43e7-a059-effd68b0593a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.421626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bed4c0-3e9f-43e7-a059-effd68b0593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.447957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-70bed4c0-3e9f-43e7-a059-effd68b0593a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:46.451472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-70bed4c0-3e9f-43e7-a059-effd68b0593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.525179 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1034/2064] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:46.561068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:46.562956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.633077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f024d42-adfd-4b33-8806-11f24e9d36a9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:46.637104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:46.637860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.638215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.638563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.650469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:46.651895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec74b18-4c2c-4aa1-9964-6aebdf676b4a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:46.662613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1031/2065] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:46.886461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7a86a-0c26-46a9-b75d-b4a5ccc956d5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:46.886851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7a86a-0c26-46a9-b75d-b4a5ccc956d5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:46.887250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7a86a-0c26-46a9-b75d-b4a5ccc956d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:46.910073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7ff7a86a-0c26-46a9-b75d-b4a5ccc956d5 None None] MFA Rules not processed for user `74581e9c5b27401686b40b2312a6f2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:46.911688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7ff7a86a-0c26-46a9-b75d-b4a5ccc956d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:46.936285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:46.982731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1035/2066] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:48.344357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:48.344893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:48.345142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:48.347117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1032/2067] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:25:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:25:48.357007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:48.358699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:48.359104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:48.408566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:48.412750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:48.646798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:48.648231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:48.666641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1036/2068] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:48.682506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:48.691729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:48.720036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58794a7e-d121-4e0d-90d4-da4d97f9b0ea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:48.735753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:48.755446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:48.758469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:48.760624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:48.761843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:48.787778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57e339-0ce2-469a-9951-41f77eb7ef9b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:48.796181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:48.797185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:48.797603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:48.797976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:48.798461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:48.800262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf1223-44f0-4c80-ab00-9dde35a5a96d service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:48.800811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:48.823416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1037/2069] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:48.998189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:48.999695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:49.015285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1033/2070] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:50.555500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:50.555500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:50.555500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:50.582557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] MFA Rules not processed for user `857d941f699143ecbf25d842fb1f9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:50.583830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:50.660111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1038/2071] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:50.671018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:50.673447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:50.684407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2658-2e3a-4734-91a7-c77be61f839a service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:50.688508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:50.689173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:50.689465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:50.689747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:50.700748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:50.702340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd6003-5ef1-4cb4-9cdc-723f15687d91 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:50.713611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1034/2072] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:51.519757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:51.521175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:51.536170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7b26c-f3e0-4e4a-ace6-ab63e6964a28 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:51.540703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:51.541584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:51.541867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:51.542141 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] PATH_INFO: `/v3/users/b3ccc67ded4144559aa3f84fd4a598f7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:51.543392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ccc67ded4144559aa3f84fd4a598f7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:51.545341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:51.593707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:51.609060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ccc67ded4144559aa3f84fd4a598f7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:51.622079 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ccc67ded4144559aa3f84fd4a598f7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:51.634073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ccc67ded4144559aa3f84fd4a598f7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:51.642808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ccc67ded4144559aa3f84fd4a598f7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:51.643520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ccc67ded4144559aa3f84fd4a598f7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:51.651465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1039/2073] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:51 2022] DELETE /identity/v3/users/b3ccc67ded4144559aa3f84fd4a598f7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:51.656119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:51.656707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:51.657000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:51.675790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:51.677632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:51.746113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1035/2074] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:51.756036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:51.758026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:51.831003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e59c1-fe06-4260-abb3-bd8a0d3ea184 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:51.834747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:51.835532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:51.835885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:51.836227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:51.847372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:51.849249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271994fc-d289-4876-9fcd-3a5f8105ed89 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:51.859476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1040/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:52.155203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:52.157142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:52.170089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd2644-8dda-42af-8d19-f7036c5700bd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:52.176006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:52.177101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:52.177390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:52.177668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] PATH_INFO: `/v3/projects/dd1ae2f16f2d42eea3d9797a8fc7fae7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:52.178905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1ae2f16f2d42eea3d9797a8fc7fae7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:52.180886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:52.263590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:52.264602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:52.265302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1ae2f16f2d42eea3d9797a8fc7fae7', 'request_id': 'req-00393cee-c990-495f-a8c3-520a7e84ec45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.265415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:52.278777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1ae2f16f2d42eea3d9797a8fc7fae7', 'request_id': 'req-00393cee-c990-495f-a8c3-520a7e84ec45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.288362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1ae2f16f2d42eea3d9797a8fc7fae7', 'request_id': 'req-00393cee-c990-495f-a8c3-520a7e84ec45'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.296811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:52.298987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] MFA Rules not processed for user `857d941f699143ecbf25d842fb1f9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:52.300536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:52.302470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] The token cache is being invalidate because project dd1ae2f16f2d42eea3d9797a8fc7fae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:52.303131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.304523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.312838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1036/2076] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:52 2022] DELETE /identity/v3/projects/dd1ae2f16f2d42eea3d9797a8fc7fae7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:52.316707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:52.318633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:52.460685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00393cee-c990-495f-a8c3-520a7e84ec45 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:52.466522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1041/2077] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:25:52.467980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:52.469583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:52.469969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:52.470364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] PATH_INFO: `/v3/users/aa3fbd14b85b445999be79cc98484ec1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:52.472679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3fbd14b85b445999be79cc98484ec1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:52.477023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:52.553268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3fbd14b85b445999be79cc98484ec1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.553847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3fbd14b85b445999be79cc98484ec1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.568941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3fbd14b85b445999be79cc98484ec1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.588727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3fbd14b85b445999be79cc98484ec1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.601710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3fbd14b85b445999be79cc98484ec1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:52.613241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1037/2078] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:52 2022] DELETE /identity/v3/users/aa3fbd14b85b445999be79cc98484ec1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:53.815409 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:53.829250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd5135-1c30-48a1-a950-5a12ec782046 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:53.898023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:53.899024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:53.899417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:53.899924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] PATH_INFO: `/v3/projects/269d2ba7b4b54ae7aa6446c1d056a6a9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:53.901350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=269d2ba7b4b54ae7aa6446c1d056a6a9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:53.903664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:53.909810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:53.983802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269d2ba7b4b54ae7aa6446c1d056a6a9', 'request_id': 'req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:53.991585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269d2ba7b4b54ae7aa6446c1d056a6a9', 'request_id': 'req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.002944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269d2ba7b4b54ae7aa6446c1d056a6a9', 'request_id': 'req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.024397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] The token cache is being invalidate because project 269d2ba7b4b54ae7aa6446c1d056a6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:54.025591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.026361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.031936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1042/2079] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:53 2022] DELETE /identity/v3/projects/269d2ba7b4b54ae7aa6446c1d056a6a9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:54.033188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:54.034688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:54.059658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8d96c-87f8-4ead-9d78-677fff03d084 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:54.146015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:54.147026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:54.147476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:54.147815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] PATH_INFO: `/v3/users/8ec8127acd53414d8b428dab88ab4c7f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:54.149361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec8127acd53414d8b428dab88ab4c7f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:54.153017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:54.177337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:54.215916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec8127acd53414d8b428dab88ab4c7f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.216604 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec8127acd53414d8b428dab88ab4c7f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.226183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec8127acd53414d8b428dab88ab4c7f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.255691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec8127acd53414d8b428dab88ab4c7f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.263301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8f367bee-de10-4c95-9886-251ac8d6ff3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec8127acd53414d8b428dab88ab4c7f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.272149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1038/2080] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:54 2022] DELETE /identity/v3/users/8ec8127acd53414d8b428dab88ab4c7f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:54.731470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:54.733132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:54.755993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ecba8-2c4d-47d5-9b8c-5a1552f038fb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:54.840768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:54.841645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:54.841929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:54.842209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] PATH_INFO: `/v3/projects/2188796a964a4e70b06c3a3e04af2767` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:54.843606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2188796a964a4e70b06c3a3e04af2767)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:54.846196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:54.900859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2188796a964a4e70b06c3a3e04af2767', 'request_id': 'req-04c3e928-88af-415a-a017-4b152131883e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.907904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2188796a964a4e70b06c3a3e04af2767', 'request_id': 'req-04c3e928-88af-415a-a017-4b152131883e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.913638 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2188796a964a4e70b06c3a3e04af2767', 'request_id': 'req-04c3e928-88af-415a-a017-4b152131883e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.928096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] The token cache is being invalidate because project 2188796a964a4e70b06c3a3e04af2767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:54.928388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.929002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-04c3e928-88af-415a-a017-4b152131883e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:54.930540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1043/2081] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:54 2022] DELETE /identity/v3/projects/2188796a964a4e70b06c3a3e04af2767 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:25:54.938216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:54.939399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:54.939879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:54.961469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:54.962684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:55.026032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1039/2082] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:55.036226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.036518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.036800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.055428 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:55.056658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:55.064584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1044/2083] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:55.068756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.078965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8bf66-4388-4a41-a65b-22d098567e53 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.083947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.084646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.084936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.085212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.086096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:55.086558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.088325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.113981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1040/2084] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:55.118033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.128155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff07fc2-6197-4837-8338-4408c34db1fe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.134365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.134962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.135733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.135979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.136765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.138946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.152568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:55.178148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1045/2085] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:55.181950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.192765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d639b7-9a21-4ef0-b521-bcfdc5634a8e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.273168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.274842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.274842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.277085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.278216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.283497 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.315303 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1041/2086] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:55.318343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.329468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48054c25-6039-442c-baa1-699ef9247a75 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.334278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.334954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.335276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.335556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.336298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.337891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.351572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1046/2087] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:55.353458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.364085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de14-b2c6-4205-b16f-f5e108b33a04 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.368721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.369397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.369683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.369958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.370696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.372369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed68a33-8a8a-4b51-a528-f92105dfa4c5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.383921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1042/2088] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:55.387882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.399027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc71278-3d0e-494a-b482-37c480834ceb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.403926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.404826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.405116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.405400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] PATH_INFO: `/v3/projects/1bf1c1d1363f4757b65277780a49a285/users/0fcb5600d6e744dda46f0cdee012cda7/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.421631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf1c1d1363f4757b65277780a49a285, user_id=0fcb5600d6e744dda46f0cdee012cda7, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.424466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.437400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1047/2089] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:55 2022] PUT /identity/v3/projects/1bf1c1d1363f4757b65277780a49a285/users/0fcb5600d6e744dda46f0cdee012cda7/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:55.448964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.449265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.449553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.456207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:55.482995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] MFA Rules not processed for user `0fcb5600d6e744dda46f0cdee012cda7`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:55.484312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:55.550412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1043/2090] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:55.554753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.615347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fad63-f091-4098-81a6-0229999841b9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.619897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.620550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.620835 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.621110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.621802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.623760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.649185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1048/2091] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:25:55.653335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.664077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd5e72-c04a-4415-a7c1-03f6f8496078 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.720137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.720817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.722310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.722310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.722310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.724043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.750173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1044/2092] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:55.752249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.762882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b1a2b-d031-407a-b8cd-d15ef8415d40 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.767726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.768672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.768895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.769244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.770208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.771953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.785513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1049/2093] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:55.789557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.800889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39078ebf-7adc-4499-9b1e-58b126d8aee3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.805721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.806727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.807090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.807466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] PATH_INFO: `/v3/projects/930f650248d64b00a790957805c165af/users/3ac3700fe9aa4258af2030978ef20a95/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.826099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930f650248d64b00a790957805c165af, user_id=3ac3700fe9aa4258af2030978ef20a95, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.830182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.847889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1045/2094] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:55 2022] PUT /identity/v3/projects/930f650248d64b00a790957805c165af/users/3ac3700fe9aa4258af2030978ef20a95/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:55.852386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.863418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae8718-035a-437e-9513-1d2f402f75d8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.921287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.922496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.922783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.923062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.923834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.925444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e34a43-b98f-4f31-8772-bc7fd81ed2a9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.935429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1050/2095] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:55.940817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:55.955256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48bc65-7dc8-459a-8fc1-4a60ddc15899 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:55.961025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:55.962439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.962725 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.963009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] PATH_INFO: `/v3/projects/930f650248d64b00a790957805c165af/users/3ac3700fe9aa4258af2030978ef20a95/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:55.965385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930f650248d64b00a790957805c165af, user_id=3ac3700fe9aa4258af2030978ef20a95, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:55.968883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:55.987261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1046/2096] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:55 2022] PUT /identity/v3/projects/930f650248d64b00a790957805c165af/users/3ac3700fe9aa4258af2030978ef20a95/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:55.994887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:55.995197 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:55.995524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:56.029425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] MFA Rules not processed for user `3ac3700fe9aa4258af2030978ef20a95`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:56.030963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:56.046489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:56.108734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1051/2097] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:56.121622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:56.123386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:56.196347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:56.211480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422ff0e-edb3-43c2-adb1-c0ed07dcea61 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:56.215775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:56.223095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:56.223489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:56.224082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:56.234140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:56.236234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe4d85-6d89-43c1-a639-d3ff47effbc1 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:56.247998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1047/2098] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:25:56.853483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:56.947324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4c82b-efd7-4296-860a-2acbf9b33a70 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:56.955013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:56.956535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:56.957230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:56.957743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] PATH_INFO: `/v3/users/0fcb5600d6e744dda46f0cdee012cda7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:56.959430 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fcb5600d6e744dda46f0cdee012cda7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:56.962011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:57.052686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fcb5600d6e744dda46f0cdee012cda7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.068521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fcb5600d6e744dda46f0cdee012cda7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.077554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fcb5600d6e744dda46f0cdee012cda7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.087350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fcb5600d6e744dda46f0cdee012cda7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.087978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fcb5600d6e744dda46f0cdee012cda7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.103958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1052/2099] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:56 2022] DELETE /identity/v3/users/0fcb5600d6e744dda46f0cdee012cda7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:57.105873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:57.106275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:57.106669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:57.127579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:57.128812 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:57.211095 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1048/2100] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:57.230208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:57.231815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:57.276472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:57.304866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652a919-fd34-426e-8cbb-66bd29cda4b7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:57.308616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:57.309281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:57.309574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:57.309858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:57.321786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:57.323351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:57.335386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1053/2101] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:25:57.705320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:57.706776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:57.735301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ee792-b6df-43e3-a9c5-153defee9d68 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:57.740459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:57.741729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:57.741729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:57.742126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] PATH_INFO: `/v3/projects/1bf1c1d1363f4757b65277780a49a285` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:57.743441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf1c1d1363f4757b65277780a49a285)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:57.745418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:57.797440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf1c1d1363f4757b65277780a49a285', 'request_id': 'req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.808522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf1c1d1363f4757b65277780a49a285', 'request_id': 'req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.815668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf1c1d1363f4757b65277780a49a285', 'request_id': 'req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.829219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] The token cache is being invalidate because project 1bf1c1d1363f4757b65277780a49a285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:57.829511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.830181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.831749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1049/2102] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:57 2022] DELETE /identity/v3/projects/1bf1c1d1363f4757b65277780a49a285 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:57.835736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:57.845621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea504de0-8fef-43ab-bfba-f641fecfffa7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:57.905228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:57.907165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:57.907489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:57.907782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] PATH_INFO: `/v3/users/3ac3700fe9aa4258af2030978ef20a95` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:57.909022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac3700fe9aa4258af2030978ef20a95)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:57.910989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:57.961843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac3700fe9aa4258af2030978ef20a95'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.973817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac3700fe9aa4258af2030978ef20a95'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.983431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac3700fe9aa4258af2030978ef20a95'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.992522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac3700fe9aa4258af2030978ef20a95'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.993085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bc680db6-d6e7-4fb8-b712-84119c7b7dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac3700fe9aa4258af2030978ef20a95'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:57.999363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1054/2103] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:25:57 2022] DELETE /identity/v3/users/3ac3700fe9aa4258af2030978ef20a95 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:58.462400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:58.464165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:58.477098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbc4b4-5151-43e4-bac6-903dc1f3d7ed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:58.543071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:58.543970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:58.544254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:58.544532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] PATH_INFO: `/v3/projects/930f650248d64b00a790957805c165af` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:58.545713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] RBAC: Authorizing `identity:delete_project(project_id=930f650248d64b00a790957805c165af)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:58.547654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:58.558983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:58.602827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930f650248d64b00a790957805c165af', 'request_id': 'req-4f4675fc-3724-486e-b02d-c5a1a8fd445e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:58.623914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930f650248d64b00a790957805c165af', 'request_id': 'req-4f4675fc-3724-486e-b02d-c5a1a8fd445e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:58.632026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930f650248d64b00a790957805c165af', 'request_id': 'req-4f4675fc-3724-486e-b02d-c5a1a8fd445e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:58.644228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] The token cache is being invalidate because project 930f650248d64b00a790957805c165af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:25:58.644595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:58.645461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4f4675fc-3724-486e-b02d-c5a1a8fd445e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:25:58.647628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1050/2104] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:25:58 2022] DELETE /identity/v3/projects/930f650248d64b00a790957805c165af => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:58.652172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:58.652462 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:58.652745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:58.674008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:58.675262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:58.727153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:25:58.747280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1055/2105] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:58.750766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:58.751061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:58.751387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:58.769386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:58.772702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:58.781080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1051/2106] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:58.785405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:58.795859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d2c34-b7d9-435a-895f-b01c02a69288 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:58.802354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:58.803185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:58.803574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:58.803925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:58.804821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:25:58.805208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:58.806904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:58.834695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1056/2107] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:25:58.839010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:58.850553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804e8b6-f358-468b-8ec9-210d4260bead None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:58.856820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:58.857667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:58.858046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:58.858428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:58.859310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:58.861986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:58.899174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1052/2108] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:25:58 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:58.900921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:58.912697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b982-13c5-46b1-838a-5920c1c157bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:59.006572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:59.007604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.007858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.008133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.008847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:59.011049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:59.032812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1057/2109] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:25:58 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:25:59.036926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:59.049980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3778f-9253-4e14-a82f-8d96b7ad1d57 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:59.055910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:59.056801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.057180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.057507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.058532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:59.060389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:59.074289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1053/2110] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:59.078261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:59.088581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241f4cd-502c-48ea-b57f-a58014d06f55 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:59.093367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:59.094776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.095096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.095412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.096161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:59.097785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f86509-1c18-4653-9e0a-677061dde00a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:59.108294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1058/2111] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:25:59.113032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:59.124260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027556-4e07-455e-9d85-1cbcf9ea6a97 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:59.128999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:59.129964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.130326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.130669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] PATH_INFO: `/v3/projects/93eff55bb6eb41e7b057580f1f0b4630/users/8f72cc0da2874545bbbe44c842fbd20a/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.146403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eff55bb6eb41e7b057580f1f0b4630, user_id=8f72cc0da2874545bbbe44c842fbd20a, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:59.149240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:59.164186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1054/2112] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:25:59 2022] PUT /identity/v3/projects/93eff55bb6eb41e7b057580f1f0b4630/users/8f72cc0da2874545bbbe44c842fbd20a/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:25:59.174691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.174985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.175321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.201741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] MFA Rules not processed for user `8f72cc0da2874545bbbe44c842fbd20a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:25:59.203014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:59.279348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1059/2113] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:25:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:25:59.288970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:25:59.290726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:25:59.361565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61c5db-cadd-4e91-9dbd-71b690932f4e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:25:59.367679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:25:59.368448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:25:59.368795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:25:59.369147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:25:59.380949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:25:59.382745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:25:59.410971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1055/2114] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:25:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:06.825836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:06.827649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:06.899320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:06.916601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602ed93-8a6e-487a-bb0c-7147cf6b6259 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:06.929701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:06.929701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:06.929701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:06.929701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] PATH_INFO: `/v3/users/213ac803c2cb4cbfb4d73ec4fb07d552` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:06.930057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213ac803c2cb4cbfb4d73ec4fb07d552)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:06.930057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:06.979758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213ac803c2cb4cbfb4d73ec4fb07d552'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:06.992927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213ac803c2cb4cbfb4d73ec4fb07d552'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.000368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213ac803c2cb4cbfb4d73ec4fb07d552'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.007677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213ac803c2cb4cbfb4d73ec4fb07d552'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.008322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8b483ff3-0f1c-4b58-8374-90228ce5a485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213ac803c2cb4cbfb4d73ec4fb07d552'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.014230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1060/2115] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:06 2022] DELETE /identity/v3/users/213ac803c2cb4cbfb4d73ec4fb07d552 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:07.284423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.299741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e569ee-f60a-41ce-866a-e8a509310035 service glance] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.353601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:07.378635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.379543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.379836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.380120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] PATH_INFO: `/v3/projects/9be64cd89e3b46738dca85e516a60d8a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.381314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be64cd89e3b46738dca85e516a60d8a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.383248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.433067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be64cd89e3b46738dca85e516a60d8a', 'request_id': 'req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.441749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be64cd89e3b46738dca85e516a60d8a', 'request_id': 'req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.448077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be64cd89e3b46738dca85e516a60d8a', 'request_id': 'req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.459691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] The token cache is being invalidate because project 9be64cd89e3b46738dca85e516a60d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:07.459986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.460647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-455cbaed-bd36-4a12-a4c8-0ef56b9f9da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:07.462372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1056/2116] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:07 2022] DELETE /identity/v3/projects/9be64cd89e3b46738dca85e516a60d8a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:07.467059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.467348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.467634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.487101 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:07.488382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:07.550916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1061/2117] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:07.559769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.560091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.560407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.580231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:07.581439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:07.589611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1057/2118] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:07.593756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.603698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d525d1-2763-4f29-9f43-77dc8bdc7b47 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.608361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.609028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.609314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.609629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.610408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:07.610722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.612378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.623828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1062/2119] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:07.629063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.638365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a3ae-6ed2-4a9e-b0cd-489a79744edc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.643777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.644523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.644873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.645218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.646018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.648046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.679815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1058/2120] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:07 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:07.680976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.695553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d854c-ca26-4ac3-8279-a2e502ea4a22 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.763486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:07.773705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.774941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.775671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.776190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.777120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.779206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.813846 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1063/2121] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:07 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:07.815040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.831664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b5226-d7fb-4f7a-807f-dc9b3593fa64 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.838076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.838858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.839334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.839848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.840650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.842672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.859272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1059/2122] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:07.861945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.874567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb286e-dfb5-425b-8c89-4cdbddb92d01 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.880596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.881195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.881582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.881744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.882531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.884173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b2d67-33ce-4b78-97fb-fdbf49161a3a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.902250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1064/2123] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:07.906878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:07.919031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c25730-5018-4e05-a513-b8c9bac4dec2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:07.924437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:07.925425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.926609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.926609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] PATH_INFO: `/v3/projects/f9be25b34e534c0daaf956c54f7b48f6/users/533820c3e0fb49dbabc0c898c8558cc0/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:07.947086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9be25b34e534c0daaf956c54f7b48f6, user_id=533820c3e0fb49dbabc0c898c8558cc0, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:07.960065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:07.983839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1060/2124] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:07 2022] PUT /identity/v3/projects/f9be25b34e534c0daaf956c54f7b48f6/users/533820c3e0fb49dbabc0c898c8558cc0/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:07.994110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:07.994404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:07.994690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.025949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] MFA Rules not processed for user `533820c3e0fb49dbabc0c898c8558cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:08.027241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:08.172024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1065/2125] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:08.176719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.274231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a55a-e743-4559-a514-f0d487ec8301 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.280259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.280955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.281523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.281818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.283103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.285486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.297111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:08.320967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1061/2126] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:08 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:08.324205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.336535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738209-0e40-4ff1-982c-b1ac0ddc9397 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.422026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.422819 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.424334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.426292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.427103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.429234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.451406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1066/2127] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:08 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:08.455982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.474098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16079d67-5d3b-4bdc-a158-d1b0a6d9be59 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.485952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.485952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.485952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.485952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.486307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.486390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.506807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1062/2128] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:08.510899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.522585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4abca4-bc8c-47c5-92d2-01ef760af0f9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.527178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.528092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.528400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.528675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] PATH_INFO: `/v3/projects/d9f230582fdc44eaa378de2c915df8e6/users/d2ca736aae104bebb94a00ab014fd4e5/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.543865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f230582fdc44eaa378de2c915df8e6, user_id=d2ca736aae104bebb94a00ab014fd4e5, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.546539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.560736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1067/2129] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:08 2022] PUT /identity/v3/projects/d9f230582fdc44eaa378de2c915df8e6/users/d2ca736aae104bebb94a00ab014fd4e5/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:08.565351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.575567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ee93c-b50f-452e-b0d6-f85e3c5c6a25 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.633083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.633706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.633999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.634279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.635034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.636799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2778937-aa9e-462d-b57a-4f0fd04f12bb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.649541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1063/2130] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:08.653454 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.664024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22328d-ba48-43eb-816b-8ab6c732ff55 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.668524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.669399 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.669678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.669960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] PATH_INFO: `/v3/projects/d9f230582fdc44eaa378de2c915df8e6/users/d2ca736aae104bebb94a00ab014fd4e5/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.671927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f230582fdc44eaa378de2c915df8e6, user_id=d2ca736aae104bebb94a00ab014fd4e5, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.675035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.685341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1068/2131] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:08 2022] PUT /identity/v3/projects/d9f230582fdc44eaa378de2c915df8e6/users/d2ca736aae104bebb94a00ab014fd4e5/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:08.695895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.696191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.696476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.722959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] MFA Rules not processed for user `d2ca736aae104bebb94a00ab014fd4e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:08.724245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:08.795880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1064/2132] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:08.807730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:08.809496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:08.830324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:08.870811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d9ea9-6b1c-41b4-a41c-71e9919b9ea3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:08.874546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:08.875199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:08.875514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:08.875790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:08.885194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:08.886769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0876ac-b829-4bc1-8e8d-988ad49509ef service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:08.896472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1069/2133] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:08.989960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:09.000257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f7ef-5ca5-4fd5-b1b2-d78b0b7b4faf None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:09.004371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:09.005023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:09.005439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:09.005731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:09.070180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:09.071693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:09.076418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:09.082743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1065/2134] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:18.821565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:18.822030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:18.822139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:18.839733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:18.840919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:18.895269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1070/2135] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:18.935346 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:18.936753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.017255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b51ba-b097-48b7-9669-47950cdee355 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:19.022926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:19.023674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.023980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.024503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.040295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:19.042090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7b0c-639a-4015-b728-565358ddd94a service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:19.051829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1066/2136] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:26:19.108656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:19.177490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfd115-fd53-4e28-9a11-c0422bd6041a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:19.182145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:19.183098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.183427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.183701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] PATH_INFO: `/v3/users/533820c3e0fb49dbabc0c898c8558cc0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.185566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=533820c3e0fb49dbabc0c898c8558cc0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:19.187526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:19.237410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533820c3e0fb49dbabc0c898c8558cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.250190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533820c3e0fb49dbabc0c898c8558cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.252140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:19.260049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533820c3e0fb49dbabc0c898c8558cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.268366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533820c3e0fb49dbabc0c898c8558cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.269455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533820c3e0fb49dbabc0c898c8558cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.275690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1071/2137] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:19 2022] DELETE /identity/v3/users/533820c3e0fb49dbabc0c898c8558cc0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:19.280412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003b35-c023-4f11-8e63-3f628e62fdcd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.280710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003b35-c023-4f11-8e63-3f628e62fdcd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.281245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23003b35-c023-4f11-8e63-3f628e62fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.299697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-23003b35-c023-4f11-8e63-3f628e62fdcd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:19.300901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-23003b35-c023-4f11-8e63-3f628e62fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.365682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1067/2138] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:19.376460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:19.378196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.467506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5227086-4169-4139-a7fd-c527812b3d19 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:19.471899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:19.472561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.472870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.473148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.484579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:19.486464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:19.497478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1072/2139] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:19.646058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bef547-c104-4065-802c-9e860310595c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.646353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bef547-c104-4065-802c-9e860310595c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.646636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bef547-c104-4065-802c-9e860310595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.665777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e4bef547-c104-4065-802c-9e860310595c None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:19.667016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e4bef547-c104-4065-802c-9e860310595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.725711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1068/2140] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:19.740626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:19.742052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.797729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bef547-c104-4065-802c-9e860310595c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:19.799249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e4bef547-c104-4065-802c-9e860310595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:19.823687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9a91-c4f9-4fb5-ba34-1899a7884ead service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:19.829307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bef547-c104-4065-802c-9e860310595c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:19.829482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:19.830354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.830749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.831429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.838375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:19.840338 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:19.840843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:19.841404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] PATH_INFO: `/v3/projects/f9be25b34e534c0daaf956c54f7b48f6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:19.843144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9be25b34e534c0daaf956c54f7b48f6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:19.846567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:19.849683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:19.852828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:19.864928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1073/2141] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 15:26:19.899530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:19.916879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9be25b34e534c0daaf956c54f7b48f6', 'request_id': 'req-6cc6f83e-e746-4175-a449-996e1b751cd0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.925795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9be25b34e534c0daaf956c54f7b48f6', 'request_id': 'req-6cc6f83e-e746-4175-a449-996e1b751cd0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.937330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9be25b34e534c0daaf956c54f7b48f6', 'request_id': 'req-6cc6f83e-e746-4175-a449-996e1b751cd0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.953177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] The token cache is being invalidate because project f9be25b34e534c0daaf956c54f7b48f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:19.953880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.954917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6cc6f83e-e746-4175-a449-996e1b751cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:19.957579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1069/2142] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:19 2022] DELETE /identity/v3/projects/f9be25b34e534c0daaf956c54f7b48f6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:19.962875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:19.974344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db952b40-05bc-4162-9375-68880cd289eb service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.037257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.038147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.038481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.038869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] PATH_INFO: `/v3/users/d2ca736aae104bebb94a00ab014fd4e5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.040119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ca736aae104bebb94a00ab014fd4e5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.042031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.068494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=1,keystone:DELETE=2,keystone:SELECT=97 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:20.101186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ca736aae104bebb94a00ab014fd4e5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.115833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ca736aae104bebb94a00ab014fd4e5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.123587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ca736aae104bebb94a00ab014fd4e5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.127539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.127935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.128310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.131167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ca736aae104bebb94a00ab014fd4e5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.131919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ca736aae104bebb94a00ab014fd4e5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.139535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1074/2143] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:19 2022] DELETE /identity/v3/users/d2ca736aae104bebb94a00ab014fd4e5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:20.159132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:20.160458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.217211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1070/2144] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:20.230550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.232063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.318886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60d092-8acc-4128-b8c9-f9a44d16823f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.325138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.325829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.326119 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.327016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.338139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.339831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85096f-a3b5-4cd2-8b8a-150addc5d9f7 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.349319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1075/2145] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:26:20.451383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.454878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.575508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc467b-e193-44f1-a0af-baae1aa12189 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.581342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.582436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.582674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.582954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] PATH_INFO: `/v3/projects/d9f230582fdc44eaa378de2c915df8e6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.584365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f230582fdc44eaa378de2c915df8e6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.586860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.654943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f230582fdc44eaa378de2c915df8e6', 'request_id': 'req-122dd37e-2248-4d6c-9cc4-43651ab5892d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.663876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f230582fdc44eaa378de2c915df8e6', 'request_id': 'req-122dd37e-2248-4d6c-9cc4-43651ab5892d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.670399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f230582fdc44eaa378de2c915df8e6', 'request_id': 'req-122dd37e-2248-4d6c-9cc4-43651ab5892d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.675447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:20.686593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] The token cache is being invalidate because project d9f230582fdc44eaa378de2c915df8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:20.686973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.688526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-122dd37e-2248-4d6c-9cc4-43651ab5892d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:20.690363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1071/2146] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:20 2022] DELETE /identity/v3/projects/d9f230582fdc44eaa378de2c915df8e6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:26:20.695311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885ad9-1c99-43c1-b388-79e956cce573 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.695606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885ad9-1c99-43c1-b388-79e956cce573 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.695886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6885ad9-1c99-43c1-b388-79e956cce573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.715809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b6885ad9-1c99-43c1-b388-79e956cce573 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:20.716987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b6885ad9-1c99-43c1-b388-79e956cce573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.753556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.753964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.754125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.775721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:20.777188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.810567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1076/2147] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:20.818662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.819024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.819401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.840407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:20.841716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.844232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1072/2148] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:20.850630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1077/2149] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:20.854857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.856474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.857984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:20.867203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c90fde-6c63-49cd-8289-d16822904d76 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.873341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.874198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.874514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.874795 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.876115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:20.876425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.878646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.878923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:20.894973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1073/2150] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:20.899042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.912053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5f4c-15a7-4140-a6b2-e232a81fecab admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.916990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.917648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.917941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.918223 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.918918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.920855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.942668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8617003-6254-41c7-a33c-aec15f53be6d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:20.948213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:20.948875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:20.949162 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:20.949440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:20.950893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1074/2151] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:20.956152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:20.959137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:20.960764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:20.969389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1078/2152] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 15:26:21.064130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1e94-3ba5-45e4-864d-3ca37056d151 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.073115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.074198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.074688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.075138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.076273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.079168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.109711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1075/2153] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:20 2022] POST /identity/v3/users => generated 282 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:21.113809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.133406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d031f-7917-4324-9a44-3f884d2c3413 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.138784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.139688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.140123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.141793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.142703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.145028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.159129 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1079/2154] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:21.163354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.181480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473aa2e7-cbe7-4477-a327-8a168f6a02bf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.193131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.193911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.194336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.194688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.196749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.198561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cde00-ea39-46aa-ba3a-8dc0b3340b52 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.225028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1076/2155] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:21.230157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.254228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e274d0b-5feb-4a35-ae5a-5411e1d8d7b0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.270050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.272330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.274657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.274976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] PATH_INFO: `/v3/projects/a8ef882f083f4edf9906b3063e6ef947/users/f50ee399040d40849167eab213b728c9/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.294026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ef882f083f4edf9906b3063e6ef947, user_id=f50ee399040d40849167eab213b728c9, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.296808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.347180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1080/2156] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:21 2022] PUT /identity/v3/projects/a8ef882f083f4edf9906b3063e6ef947/users/f50ee399040d40849167eab213b728c9/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:21.357986 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.358421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.358804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.397139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] MFA Rules not processed for user `f50ee399040d40849167eab213b728c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:21.398361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:21.430599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:21.470644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1077/2157] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:21.475037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.542782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782390aa-afbd-4d61-a9cf-176b3e88093d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.548223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.548887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.549171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.549613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.550324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.552348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.582093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1081/2158] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:21 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:21.586222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.597521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bd1b2-46f1-4e90-8909-da9a62fce4cd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.667235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.667947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.668474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.668474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.669177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.671123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.695040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1078/2159] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:21 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:21.696216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.706369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455a573-ecfb-4ee5-bd36-290ce9089a1d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.710865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.711557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.711844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.712137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.712856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.714432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.727519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1082/2160] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:21.729856 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.741550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32123374-3e77-4a41-a073-adf7338b5934 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.746381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.751298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.751298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.751298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] PATH_INFO: `/v3/projects/9f07d67725494b14a1a246acff21f1a3/users/bd510d1f6cd64296a07bbc578bfc9042/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.769808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f07d67725494b14a1a246acff21f1a3, user_id=bd510d1f6cd64296a07bbc578bfc9042, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.774124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.790021 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1079/2161] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:21 2022] PUT /identity/v3/projects/9f07d67725494b14a1a246acff21f1a3/users/bd510d1f6cd64296a07bbc578bfc9042/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:21.791543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.803036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b45b1-9bbe-4df6-a53e-e17f10b70741 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.854151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:21.873075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.873075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.873075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.873075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.873417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.877975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f099f6e-47b8-461d-a05f-e26d2fa2a428 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.904499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1083/2162] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:21.906168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:21.931334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bc1a-459c-42c9-9e59-5cdd0a3dd742 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:21.946071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:21.949206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:21.949523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:21.949807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] PATH_INFO: `/v3/projects/9f07d67725494b14a1a246acff21f1a3/users/bd510d1f6cd64296a07bbc578bfc9042/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:21.961315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f07d67725494b14a1a246acff21f1a3, user_id=bd510d1f6cd64296a07bbc578bfc9042, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:21.963434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:21.987191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1080/2163] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:21 2022] PUT /identity/v3/projects/9f07d67725494b14a1a246acff21f1a3/users/bd510d1f6cd64296a07bbc578bfc9042/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:22.001324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722f71-cb2e-417b-880d-2eaf82697c25 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:22.006929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722f71-cb2e-417b-880d-2eaf82697c25 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:22.007244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f722f71-cb2e-417b-880d-2eaf82697c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:22.069747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0f722f71-cb2e-417b-880d-2eaf82697c25 None None] MFA Rules not processed for user `bd510d1f6cd64296a07bbc578bfc9042`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:22.070984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0f722f71-cb2e-417b-880d-2eaf82697c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:22.148498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1084/2164] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:22.161110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:22.163162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:22.225187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15519a27-0fc7-4948-b5b9-64c94b2b3191 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:22.229105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:22.229760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:22.230069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:22.230445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:22.241069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:22.242538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:22.258207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1081/2165] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:22.428573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7571b3-3f2f-42f6-850d-c5cfd44da7bb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:22.428808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7571b3-3f2f-42f6-850d-c5cfd44da7bb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:22.429077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7571b3-3f2f-42f6-850d-c5cfd44da7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:22.447774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7d7571b3-3f2f-42f6-850d-c5cfd44da7bb None None] MFA Rules not processed for user `857d941f699143ecbf25d842fb1f9ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:22.448993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7d7571b3-3f2f-42f6-850d-c5cfd44da7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:22.513798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1085/2166] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:22.711191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:22.741857 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:22.782276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa64fb-b821-4a03-9a5f-2db63b47d318 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:22.790454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:22.791472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:22.791759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:22.792154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] PATH_INFO: `/v3/users/f50ee399040d40849167eab213b728c9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:22.796623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50ee399040d40849167eab213b728c9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:22.799197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:22.848367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50ee399040d40849167eab213b728c9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:22.849028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50ee399040d40849167eab213b728c9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:22.855871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50ee399040d40849167eab213b728c9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:22.869184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50ee399040d40849167eab213b728c9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:22.876739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50ee399040d40849167eab213b728c9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:22.882834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1082/2167] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:22 2022] DELETE /identity/v3/users/f50ee399040d40849167eab213b728c9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:22.887393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:22.887856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:22.887948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:22.905403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:22.906869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:22.965510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1086/2168] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:22.975157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:22.976901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.036575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96ee7d-4913-49f0-83a1-597e0f6dff49 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:23.044043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:23.044735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.045132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.045286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.060794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:23.062435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:23.077862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1083/2169] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:23.336070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:23.338073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.360598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec419-5abf-447e-9cd5-cac047935897 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:23.366622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:23.367595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.367966 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.368311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] PATH_INFO: `/v3/users/242a289932b34e06ac763d96c6315037` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.370503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242a289932b34e06ac763d96c6315037)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:23.373046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:23.382449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:23.441695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242a289932b34e06ac763d96c6315037'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.463558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242a289932b34e06ac763d96c6315037'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.475464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242a289932b34e06ac763d96c6315037'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.490133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242a289932b34e06ac763d96c6315037'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.491723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:23.491877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-18530cbc-da49-4a31-9214-a706e8348227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242a289932b34e06ac763d96c6315037'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.494792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.499809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1087/2170] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:23 2022] DELETE /identity/v3/users/242a289932b34e06ac763d96c6315037 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:23.504767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.505104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.505392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.524078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:23.525433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.534171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8de-6d07-4326-b60c-67c889675cf3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:23.586884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:23.595287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1088/2171] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:23.602203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:23.604189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.604338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:23.604607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.605145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] PATH_INFO: `/v3/projects/a8ef882f083f4edf9906b3063e6ef947` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.605670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.607144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ef882f083f4edf9906b3063e6ef947)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:23.609388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:23.661607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ef882f083f4edf9906b3063e6ef947', 'request_id': 'req-e1bef336-9ad0-42b2-a21a-180eb64b45f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.670149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af57e4-e14b-45e4-8faa-198452a2ab14 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:23.675992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ef882f083f4edf9906b3063e6ef947', 'request_id': 'req-e1bef336-9ad0-42b2-a21a-180eb64b45f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.687283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ef882f083f4edf9906b3063e6ef947', 'request_id': 'req-e1bef336-9ad0-42b2-a21a-180eb64b45f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.714269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] The token cache is being invalidate because project a8ef882f083f4edf9906b3063e6ef947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:23.714572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.715577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.717359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1084/2172] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:23 2022] DELETE /identity/v3/projects/a8ef882f083f4edf9906b3063e6ef947 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:23.720973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:23.722357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.741354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:23.752841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:23.753628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.753993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.754422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.759615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:23.806884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef336-9ad0-42b2-a21a-180eb64b45f5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:23.812094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:23.813521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:23.813920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:23.814204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] PATH_INFO: `/v3/users/bd510d1f6cd64296a07bbc578bfc9042` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:23.815468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd510d1f6cd64296a07bbc578bfc9042)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:23.817388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:23.827843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:23.829277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:23.839002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1089/2173] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:23.864485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd510d1f6cd64296a07bbc578bfc9042'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.865156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd510d1f6cd64296a07bbc578bfc9042'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.872511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd510d1f6cd64296a07bbc578bfc9042'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.885915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd510d1f6cd64296a07bbc578bfc9042'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.892961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd510d1f6cd64296a07bbc578bfc9042'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:23.894791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:23.900762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1085/2174] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:23 2022] DELETE /identity/v3/users/bd510d1f6cd64296a07bbc578bfc9042 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:26:24.022925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.024780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.041259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a100-4459-47c5-b6bb-e293ce3e33f0 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.108493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.109374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.110222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.110530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] PATH_INFO: `/v3/projects/56c6b45912e64ad691232fc05df8c123` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.111762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c6b45912e64ad691232fc05df8c123)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.113721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.202448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.205009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.205834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c6b45912e64ad691232fc05df8c123', 'request_id': 'req-4b5b866a-392e-4096-b89b-e08fa2007a2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.213130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c6b45912e64ad691232fc05df8c123', 'request_id': 'req-4b5b866a-392e-4096-b89b-e08fa2007a2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.221948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c6b45912e64ad691232fc05df8c123', 'request_id': 'req-4b5b866a-392e-4096-b89b-e08fa2007a2f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.237613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] The token cache is being invalidate because project 56c6b45912e64ad691232fc05df8c123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:24.237967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.241493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4b5b866a-392e-4096-b89b-e08fa2007a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.241572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b33c9-a05a-4402-9cb3-0e3f4263f1e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.244071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1090/2175] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:24 2022] DELETE /identity/v3/projects/56c6b45912e64ad691232fc05df8c123 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:24.247574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5182-1590-4063-a425-9d59a6198c79 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.247949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5182-1590-4063-a425-9d59a6198c79 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.249813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa5182-1590-4063-a425-9d59a6198c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.286604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08aa5182-1590-4063-a425-9d59a6198c79 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:24.288042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08aa5182-1590-4063-a425-9d59a6198c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.322318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:24.343247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.345378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.345378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.345378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] PATH_INFO: `/v3/projects/9f07d67725494b14a1a246acff21f1a3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.346081 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f07d67725494b14a1a246acff21f1a3)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.349281 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.428769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1091/2176] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:24.430479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.430785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.431906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.447903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f07d67725494b14a1a246acff21f1a3', 'request_id': 'req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.456927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:24.459019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.460903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f07d67725494b14a1a246acff21f1a3', 'request_id': 'req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.469721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f07d67725494b14a1a246acff21f1a3', 'request_id': 'req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.484878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] The token cache is being invalidate because project 9f07d67725494b14a1a246acff21f1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:24.485315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.486229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05a85d6c-6e0c-4e18-aac2-b16e6fec71fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:24.488164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1086/2177] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:24 2022] DELETE /identity/v3/projects/9f07d67725494b14a1a246acff21f1a3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:24.494178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b22b9-6bb6-40d2-abc2-4aea4d0044c3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.494900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b22b9-6bb6-40d2-abc2-4aea4d0044c3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.495198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b22b9-6bb6-40d2-abc2-4aea4d0044c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.521587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-117b22b9-6bb6-40d2-abc2-4aea4d0044c3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:24.522850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-117b22b9-6bb6-40d2-abc2-4aea4d0044c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.544860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1092/2178] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:24.552123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.562082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62d070-5da9-4265-b610-48aa76893d84 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.567841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.568521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.568806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.569083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.570188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:24.570505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.572170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.585779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1093/2179] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:24.590206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.595398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1087/2180] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:24.596892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.597214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.597501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.603747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7516c-df1b-40c3-a80a-5aa030bc1f17 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.609138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.609866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.610147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.610498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.612128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.615959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.618989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:24.620250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:24.628753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1088/2181] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:24.633152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.645200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cb357-79a3-41b0-ae09-15176263bca9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.649018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1094/2182] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:24.652812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.653707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.654381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.654664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.654940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.656002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:24.656402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.657985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.675521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1089/2183] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:26:24.676390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.686677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a6dd9-ef5c-482c-9bc3-d20378520f79 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.693711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:24.728358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0e4ea-25db-4064-8cbd-75cf9e57b7f2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.733254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.733951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.734241 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.734710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.735466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.737570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.760172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.760876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.761225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.761539 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.762333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.764509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1095/2184] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:24.767752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.770227 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.784994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151d22-43fd-4d0e-92e7-f336d3a4c25e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.790210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.791027 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.791356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.791646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.792396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.794009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.804812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1096/2185] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:24.808856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.810375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1090/2186] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:24.815351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.826960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1d273-5ea2-443c-af40-5728c2307ee2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.832196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:24.888408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b016c9-6e85-4974-a2e5-1d816071a729 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.894139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.895023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.895573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.895919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.896629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.898567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.899825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.900507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.900804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.901091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.901834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.903452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.914869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1097/2187] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:24.919306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.924591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1091/2188] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:24.929090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.933738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc99fc-cabd-490b-a62b-a3ad56b4a412 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.939000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.939950 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.940209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6740e-aba6-4661-85c6-8bc7a0fd5cc8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.940315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.940508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] PATH_INFO: `/v3/projects/0805ca8c35a34d0dbf0f1e7ac459c517/users/b57356f7b4e748438b89d69663bb99ee/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.945961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:24.946645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.946953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.947244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:24.948003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.950036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.956339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0805ca8c35a34d0dbf0f1e7ac459c517, user_id=b57356f7b4e748438b89d69663bb99ee, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:24.959340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798e92b-6999-43c0-8619-90a17d20ba3c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:24.961391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1092/2189] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:24.965765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:24.976537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1098/2190] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:24 2022] PUT /identity/v3/projects/0805ca8c35a34d0dbf0f1e7ac459c517/users/b57356f7b4e748438b89d69663bb99ee/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:26:24.979404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98329f76-e468-46df-819c-85c35848c7b1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:24.981622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:24.982103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:24.982593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.002652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:25.003900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:25.040404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.041005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.041292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.041571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.042312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.044666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.056961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1093/2191] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:25.062144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.074941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1099/2192] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:25.075608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151d79-294b-416c-b13a-fc3bfd8739e5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.082772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.084622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.085543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.085909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.086187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] PATH_INFO: `/v3/projects/5a9e2963382c46ca9a626bafbcb87ab2/users/8723363ec34b4980956c318c042da273/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.086561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:25.091673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:25.106642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9e2963382c46ca9a626bafbcb87ab2, user_id=8723363ec34b4980956c318c042da273, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.110388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb5687-f8d1-4c01-bda6-c60063ff2ee7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.128347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1094/2193] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:25 2022] PUT /identity/v3/projects/5a9e2963382c46ca9a626bafbcb87ab2/users/8723363ec34b4980956c318c042da273/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:25.138973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.139302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.139603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.161628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc340c8-384f-4dbd-b1fc-f0688f253566 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.176677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] MFA Rules not processed for user `8723363ec34b4980956c318c042da273`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:25.178414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:25.206277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:25.226360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.227134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.227347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.227631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.259532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1095/2194] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:25.265180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.302048 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.303699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.319024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1100/2195] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:25.375551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811d7a1-ab34-4e98-ba09-0c3df41aa528 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.381041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.381876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.382332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.382522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.383382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.385724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.421197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1096/2196] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:25 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:25.423270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.433205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802cfdf-6af0-4d41-a8d9-2ae2fc374ca4 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.507040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.507826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.508178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.508517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.509417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.512906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.543637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1101/2197] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:25 2022] POST /identity/v3/users => generated 303 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:25.548640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.560769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b973c8-c4b9-4a4f-a3d6-9a2569ed4030 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.567151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.567560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.567901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.568187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.569469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.570668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.584070 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1097/2198] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:25.588507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.628269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab9612-ce82-4005-ab58-c23feabc5234 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.640868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.641917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.642274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.642482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] PATH_INFO: `/v3/projects/d88893b634624deca5d681663155bb98/users/0b63c03599764c1fb3b48b5b262a6e80/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.673978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88893b634624deca5d681663155bb98, user_id=0b63c03599764c1fb3b48b5b262a6e80, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.677120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.700726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.701198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1102/2199] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:25 2022] PUT /identity/v3/projects/d88893b634624deca5d681663155bb98/users/0b63c03599764c1fb3b48b5b262a6e80/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:25.712504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec9da-bf0d-449c-a68e-d19d679ae127 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.804442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:25.825953 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.826769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.827132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.827614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.829030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.830769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9ee56-578a-4f8b-89c8-54557b4e37aa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.842879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1098/2200] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:25.847459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:25.861603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb1f-26c4-47f1-9a9d-8bdf5356b5f7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:25.879355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:25.880665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.881151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.881518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] PATH_INFO: `/v3/projects/d88893b634624deca5d681663155bb98/users/0b63c03599764c1fb3b48b5b262a6e80/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.884085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88893b634624deca5d681663155bb98, user_id=0b63c03599764c1fb3b48b5b262a6e80, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:25.897788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:25.932422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1103/2201] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:25 2022] PUT /identity/v3/projects/d88893b634624deca5d681663155bb98/users/0b63c03599764c1fb3b48b5b262a6e80/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:25.944173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:25.944518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:25.944811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:25.999202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] MFA Rules not processed for user `0b63c03599764c1fb3b48b5b262a6e80`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:26.001322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:26.109921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1099/2202] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:26.120947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:26.123879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:26.180321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:26.198662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25246be7-2207-4138-b061-f37a52002d4d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:26.203085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:26.204348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:26.204664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:26.204903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:26.221320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:26.223681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:26.256670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1104/2203] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:26.324312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:26.326463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:26.359299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3fdde-d686-42c6-9e1c-1c3b86ad8d24 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:26.364547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:26.365755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:26.366110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:26.366403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:26.483272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:26.485477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc311ed-7748-4c83-8384-2609fda34385 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:26.496706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1100/2204] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:27.137377 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:27.138827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:27.258154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb533-6048-4b90-b203-cb5fa4b2a88a service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:27.262717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:27.263801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:27.264185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:27.264747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] PATH_INFO: `/v3/users/8f72cc0da2874545bbbe44c842fbd20a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:27.267866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f72cc0da2874545bbbe44c842fbd20a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:27.271189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:27.347029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f72cc0da2874545bbbe44c842fbd20a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:27.364037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f72cc0da2874545bbbe44c842fbd20a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:27.374219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f72cc0da2874545bbbe44c842fbd20a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:27.383613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f72cc0da2874545bbbe44c842fbd20a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:27.384393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f72cc0da2874545bbbe44c842fbd20a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:27.393028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1105/2205] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:27 2022] DELETE /identity/v3/users/8f72cc0da2874545bbbe44c842fbd20a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:27.397760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:27.398249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:27.398396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:27.434496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:27.439000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:27.523036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1101/2206] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:27.533999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:27.535530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:27.622252 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eacac78-b6a5-4650-bf00-8469ece4d384 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:27.626632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:27.627336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:27.627628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:27.627911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:27.637410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:27.638790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a530ceb-9fb5-4068-984b-b91a8b98e2f5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:27.649158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1106/2207] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:27.919841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:27.921645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:27.942744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c26e-13c8-41ac-974d-6ce620b9d55b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:27.952801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:27.953762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:27.962285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:27.963423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] PATH_INFO: `/v3/projects/93eff55bb6eb41e7b057580f1f0b4630` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:27.966253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eff55bb6eb41e7b057580f1f0b4630)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:27.968770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:27.975060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:28.097989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eff55bb6eb41e7b057580f1f0b4630', 'request_id': 'req-747c688a-7916-44ca-92a4-c7335d18fb66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:28.135841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eff55bb6eb41e7b057580f1f0b4630', 'request_id': 'req-747c688a-7916-44ca-92a4-c7335d18fb66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:28.146401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eff55bb6eb41e7b057580f1f0b4630', 'request_id': 'req-747c688a-7916-44ca-92a4-c7335d18fb66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:28.164710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] The token cache is being invalidate because project 93eff55bb6eb41e7b057580f1f0b4630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:28.165069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:28.165921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-747c688a-7916-44ca-92a4-c7335d18fb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:28.168162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1102/2208] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:27 2022] DELETE /identity/v3/projects/93eff55bb6eb41e7b057580f1f0b4630 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:28.173634 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.174211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.174494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.186543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:28.216604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:28.218078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:28.400155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1107/2209] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:28.410448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331b379-e25d-407d-8b23-dae37148612e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.410970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331b379-e25d-407d-8b23-dae37148612e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.411064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331b379-e25d-407d-8b23-dae37148612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.470941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5331b379-e25d-407d-8b23-dae37148612e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:28.474786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5331b379-e25d-407d-8b23-dae37148612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:28.501792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1103/2210] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:28.506297 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.527892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916e28e-4e6c-4722-aafa-a6a8af10c33d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.539086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.540274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.541267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.541549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.542365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:28.542690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.546069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:28.571288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1108/2211] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:28.581543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331b379-e25d-407d-8b23-dae37148612e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.592356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331b379-e25d-407d-8b23-dae37148612e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.598729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.599751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.600257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.600957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.602267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.604474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:28.640920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1104/2212] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:28 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:28.644849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.657122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cec107-e9da-407b-8560-b10a80fa5c01 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.761253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.762559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.764444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.764741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.765528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.768850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:28.800808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1109/2213] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:28 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:28.806324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.820666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204c592-58a5-42cf-9a03-c7e060851f41 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.825997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.829394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.829944 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.830379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.831479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.833453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:28.847086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1105/2214] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:28.851581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.866545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a85517-ab2b-4187-ba3f-f83e74ea55f0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.874857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.875505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.875799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.876659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.877615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.881397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff4ae2-a778-4464-bda1-1936ba450081 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:28.896623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1110/2215] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:28.900570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:28.914075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a66c57-b63d-4183-a989-7ef18b128650 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:28.939355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:28.940852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:28.941843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:28.942748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] PATH_INFO: `/v3/projects/04a162567976474fb70601e678e3abb8/users/751552d2e4fe49cda44c19f0cea679cb/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:28.989238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a162567976474fb70601e678e3abb8, user_id=751552d2e4fe49cda44c19f0cea679cb, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:28.993463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.046762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1106/2216] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:28 2022] PUT /identity/v3/projects/04a162567976474fb70601e678e3abb8/users/751552d2e4fe49cda44c19f0cea679cb/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:29.057222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.057520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.057803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.124294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] MFA Rules not processed for user `751552d2e4fe49cda44c19f0cea679cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:29.128117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:29.255471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:29.269612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:29.292070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1111/2217] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:29.394153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d9d1-005d-44d0-bcbb-fd449b4152f3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:29.400930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:29.401619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.402030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.402234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.402940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:29.404909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.444255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1107/2218] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:29 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:29.445975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:29.463073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf123a6-3df4-4d5c-97fe-54637c347913 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:29.612096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:29.612798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.613086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.613366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.614078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:29.616041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.666584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1112/2219] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:29 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:29.670728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:29.688302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06c915-8ed7-4f9e-b527-ea7d43d51771 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:29.699282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:29.700756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.701247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.701746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.702717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:29.704636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.718732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1108/2220] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:29.722142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:29.736989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d884ba-abf2-4c06-8658-3b0a883465fc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:29.744017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:29.744979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.745470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.745748 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.746509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:29.748488 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e5527-b95b-499b-a7fd-3c0974a36f62 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.771373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1113/2221] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:29.771720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:29.776428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:29.785097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158aebf-12b0-41f2-b151-ba0061b29121 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:29.791917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:29.793188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.793631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.794061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] PATH_INFO: `/v3/projects/7832794269b7497fa5ced83e015cffa2/users/98055c4510dc4cbb83daab54914438cd/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.819087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7832794269b7497fa5ced83e015cffa2, user_id=98055c4510dc4cbb83daab54914438cd, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:29.821912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:29.842119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1109/2222] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:29 2022] PUT /identity/v3/projects/7832794269b7497fa5ced83e015cffa2/users/98055c4510dc4cbb83daab54914438cd/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:29.852888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:29.853191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:29.853464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:29.883969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] MFA Rules not processed for user `98055c4510dc4cbb83daab54914438cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:29.885303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:29.983907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1114/2223] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:29.988909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.086696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdcd31-24d1-418d-a719-015c98615cdc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.092816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.094092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.094092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.094293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.095601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.097431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.151472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1110/2224] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:29 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:30.157023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.173124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8b73a-a237-4c0c-86ab-101317507bf8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.305551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.306198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.306489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.306824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.307660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.309617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.324905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:30.339536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1115/2225] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:30 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:30.340729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.351827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073ffd0-0be7-4b47-abf1-6dc3d329c71c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.357968 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.358658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.358948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.359247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.360540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.362157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.387294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1111/2226] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:30.389367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.404200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1861-956f-4c14-b331-54acd10cd4a1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.410674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.412223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.412537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.412952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] PATH_INFO: `/v3/projects/557c7212f01d4bfeb4b11164e881ff20/users/0dfc0722d35242c0b7f7ae16dac4d06e/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.432363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557c7212f01d4bfeb4b11164e881ff20, user_id=0dfc0722d35242c0b7f7ae16dac4d06e, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.436168 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.455807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1116/2227] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:30 2022] PUT /identity/v3/projects/557c7212f01d4bfeb4b11164e881ff20/users/0dfc0722d35242c0b7f7ae16dac4d06e/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:30.457174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.468221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4ae90-3163-4f39-822c-1912a23cb0db admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.530869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.531623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.531915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.532198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.532945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.534557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c27522-f8c7-4709-b0f1-860232698167 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.555701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1112/2228] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:30.562003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.576058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebaca1-8710-4228-9474-1fa090813272 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.580932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.581827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.582116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.583247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] PATH_INFO: `/v3/projects/557c7212f01d4bfeb4b11164e881ff20/users/0dfc0722d35242c0b7f7ae16dac4d06e/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.584442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557c7212f01d4bfeb4b11164e881ff20, user_id=0dfc0722d35242c0b7f7ae16dac4d06e, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:30.587286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:30.601957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1117/2229] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:30 2022] PUT /identity/v3/projects/557c7212f01d4bfeb4b11164e881ff20/users/0dfc0722d35242c0b7f7ae16dac4d06e/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:26:30.616109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.616400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.617098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.653415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] MFA Rules not processed for user `0dfc0722d35242c0b7f7ae16dac4d06e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:30.655056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:30.703844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:30.766221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1113/2230] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:30.774351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.778032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:30.870180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcd29-8186-4b0b-be2a-7d6901d58184 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:30.878433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:30.879284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:30.881380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:30.881745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:30.934413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:30.935903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:31.033602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.034971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9824e-dc3a-4fa5-9302-43fc1503d011 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.069383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1118/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:31.117108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8133db2-0a5a-4993-a343-e239aa0f7c98 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.123999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.124942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.125971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.126231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] PATH_INFO: `/v3/users/8723363ec34b4980956c318c042da273` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.131613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8723363ec34b4980956c318c042da273)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.134003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.193801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720e918-352a-4467-816d-513373670c3d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.194103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720e918-352a-4467-816d-513373670c3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.194391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720e918-352a-4467-816d-513373670c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.225747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8723363ec34b4980956c318c042da273'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:31.226529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8723363ec34b4980956c318c042da273'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:31.250226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3720e918-352a-4467-816d-513373670c3d None None] MFA Rules not processed for user `b57356f7b4e748438b89d69663bb99ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:31.251635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3720e918-352a-4467-816d-513373670c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:31.252293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8723363ec34b4980956c318c042da273'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:31.284159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8723363ec34b4980956c318c042da273'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:31.301244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca0bb5fb-3912-4d25-871e-845a768889f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8723363ec34b4980956c318c042da273'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:31.314495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1114/2232] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:30 2022] DELETE /identity/v3/users/8723363ec34b4980956c318c042da273 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:31.321537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.321537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.321739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.366345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:31.368080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:31.376071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1119/2233] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:31.387405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720e918-352a-4467-816d-513373670c3d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.393576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3720e918-352a-4467-816d-513373670c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:31.430067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:31.459169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720e918-352a-4467-816d-513373670c3d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.521895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:31.556020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1115/2234] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:31.559165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.561348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:31.567492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.568035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.568313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.568624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.569715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.571345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.605973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1120/2235] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:31 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:31.610581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.637794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fae4ea-c2bb-452f-ba6f-c20e5ac3984c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.681822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7a87-f796-485f-83b5-fe9a66430857 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.686706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.687435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.687731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.688012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.688720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.690655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.705400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.706110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.706405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.706689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.715673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1121/2236] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:31 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:31.717960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.719340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.720848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.730834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e840011-0b81-450e-bc3d-364351ad053b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.731395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1116/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:31.735996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.736768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.737118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.737486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.738294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.740032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.751735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1122/2238] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:31.756153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.767748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2333a57-0baf-46ad-b5e8-4065f59ccc7c service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.773612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.774665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.774964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.775377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] PATH_INFO: `/v3/projects/a06421484e18450fa03768d5949a0893/users/247594fc7d5d446e85e1cc017a5ecea1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.793705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06421484e18450fa03768d5949a0893, user_id=247594fc7d5d446e85e1cc017a5ecea1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.797579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.814599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1117/2239] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:31 2022] PUT /identity/v3/projects/a06421484e18450fa03768d5949a0893/users/247594fc7d5d446e85e1cc017a5ecea1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:31.818407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.829647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ffb38f-4007-48a1-944e-c53342b4ea42 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.908426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.909110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.911646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.913333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.914931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.917658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:31.950517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1123/2240] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:31.953703 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:31.972175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608abc2-0045-422f-aa9c-d188ddf9beaa admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:31.980035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:31.980950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:31.981235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:31.981507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] PATH_INFO: `/v3/projects/a06421484e18450fa03768d5949a0893/users/247594fc7d5d446e85e1cc017a5ecea1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:31.983653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06421484e18450fa03768d5949a0893, user_id=247594fc7d5d446e85e1cc017a5ecea1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:31.986510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:32.001226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1118/2241] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:31 2022] PUT /identity/v3/projects/a06421484e18450fa03768d5949a0893/users/247594fc7d5d446e85e1cc017a5ecea1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:32.091517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:32.094513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:32.107743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:32.137090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a70471d-1a4c-4c26-8ad2-643a3ebcf9b8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:32.296410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:32.297645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:32.298315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:32.298562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] PATH_INFO: `/v3/projects/5a9e2963382c46ca9a626bafbcb87ab2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:32.299909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9e2963382c46ca9a626bafbcb87ab2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:32.302043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:32.402392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9e2963382c46ca9a626bafbcb87ab2', 'request_id': 'req-799e50bd-d1d5-4ce1-bef9-3925a28771db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.416765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9e2963382c46ca9a626bafbcb87ab2', 'request_id': 'req-799e50bd-d1d5-4ce1-bef9-3925a28771db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.437050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9e2963382c46ca9a626bafbcb87ab2', 'request_id': 'req-799e50bd-d1d5-4ce1-bef9-3925a28771db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.471289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] The token cache is being invalidate because project 5a9e2963382c46ca9a626bafbcb87ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:32.472128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.483777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.490023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1124/2242] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:32 2022] DELETE /identity/v3/projects/5a9e2963382c46ca9a626bafbcb87ab2 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:32.491261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:32.501480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:32.517273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a039faa-6b7a-4626-bc26-df752bc18f8d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:32.645545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:32.648425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:32.650319 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:32.650840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] PATH_INFO: `/v3/users/0b63c03599764c1fb3b48b5b262a6e80` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:32.669267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b63c03599764c1fb3b48b5b262a6e80)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:32.669267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:32.714009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:32.771205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b63c03599764c1fb3b48b5b262a6e80'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.773989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b63c03599764c1fb3b48b5b262a6e80'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.787489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b63c03599764c1fb3b48b5b262a6e80'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.812116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b63c03599764c1fb3b48b5b262a6e80'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.827466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b63c03599764c1fb3b48b5b262a6e80'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:32.852257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1119/2243] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:32 2022] DELETE /identity/v3/users/0b63c03599764c1fb3b48b5b262a6e80 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:32.917978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:32.919930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:33.076289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e50bd-d1d5-4ce1-bef9-3925a28771db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:33.149067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:33.155009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:33.156731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.157234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.158021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.161093 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:33.233548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:33.235120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b806d25-bd45-44e5-b979-47f8c5f7ede0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:33.256871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1125/2244] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:33.273266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:33.275836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:33.381078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c54a84-53df-4f5c-b5d9-7910e3c7913f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:33.401292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:33.402890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.403151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.403488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] PATH_INFO: `/v3/projects/d88893b634624deca5d681663155bb98` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.412796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88893b634624deca5d681663155bb98)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:33.417242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:33.537514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88893b634624deca5d681663155bb98', 'request_id': 'req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:33.553481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88893b634624deca5d681663155bb98', 'request_id': 'req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:33.565961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88893b634624deca5d681663155bb98', 'request_id': 'req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:33.591034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] The token cache is being invalidate because project d88893b634624deca5d681663155bb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:33.591568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:33.592530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4fc0d91c-19c1-4b7c-b73f-50b91a39ba72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:33.594594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1120/2245] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:33 2022] DELETE /identity/v3/projects/d88893b634624deca5d681663155bb98 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:33.600287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704459-624a-4c10-8609-79ac233e7856 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.600809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704459-624a-4c10-8609-79ac233e7856 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.601049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36704459-624a-4c10-8609-79ac233e7856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.648811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-36704459-624a-4c10-8609-79ac233e7856 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:33.650181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-36704459-624a-4c10-8609-79ac233e7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:33.793213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1126/2246] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:33.801001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.801509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.801707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.831698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:33.834077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:33.845874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1121/2247] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:33.850091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36704459-624a-4c10-8609-79ac233e7856 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:33.868273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36704459-624a-4c10-8609-79ac233e7856 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:33.874673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:33.876282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.876738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.884850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.885821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:33.886147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:33.888039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:33.906697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1127/2248] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:33.912596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:33.929183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d535e2-5ae8-4776-815f-8effc00ff241 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:33.945363 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:33.946120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:33.946408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:33.946751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:33.947498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:33.949586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:33.994678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:34.026860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1122/2249] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:33 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:34.027728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.051843 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d95496-dcae-4543-85cf-120ac2a14ea8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.189057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.191031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:34.196849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.197665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.198095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.198963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.199711 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.204840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.267392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1128/2250] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:34 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:34.271612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.290736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d739f9d-a67e-4760-a16a-abcfd742af10 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.299767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.301473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.301908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.302431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.303524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.305812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.321221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f846932-1290-40ff-a2ee-bce008212055 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.322796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1129/2251] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:34.326924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.334018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.334930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.335336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.335718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.339697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:34.350883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3b905-9789-4d3b-8efb-2d47ac43761a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.368487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.369190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.369480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.369762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.370518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.372534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.378627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:34.396652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1130/2252] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:26:34.400792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.421190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dab3ea-c8c8-4c27-8721-b19122682d88 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.433608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.434549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.435150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.435150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] PATH_INFO: `/v3/projects/347d7a0c2fb94726b94437204bb1d3b8/users/b63ab3a9e56d446d95e64ed80b2ff8e1/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.463751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.465858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ac7dc-f5e7-430f-9a71-4f1dcbcd55d8 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.470211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=347d7a0c2fb94726b94437204bb1d3b8, user_id=b63ab3a9e56d446d95e64ed80b2ff8e1, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.473159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.501109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1123/2253] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:34.520087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1131/2254] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:34 2022] PUT /identity/v3/projects/347d7a0c2fb94726b94437204bb1d3b8/users/b63ab3a9e56d446d95e64ed80b2ff8e1/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:34.530147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.530940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.531247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.579992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] MFA Rules not processed for user `b63ab3a9e56d446d95e64ed80b2ff8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:34.590014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:34.721340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1124/2255] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:34.726218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.875645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952efe49-4a45-437b-b38a-588feb3e2dfa admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:34.882138 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:34.882895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:34.884085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:34.884427 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:34.885219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:34.893707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:34.945453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1132/2256] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:34 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:34.947899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:34.960109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c8d4d-ee60-4476-9eef-7896e067c892 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:35.028173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:35.091118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:35.091859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.092157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.092490 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.093283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:35.095906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:35.135502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1125/2257] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:34 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:35.140753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:35.157296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976476f-3bc0-47bb-8131-30918e411ac9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:35.186927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:35.192058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.192361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.196975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.198011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:35.204362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:35.234254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1133/2258] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:35.236527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:35.248946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be477f5-8ae9-4b39-a393-f89c32a65f9d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:35.256049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:35.257205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.257570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.257925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] PATH_INFO: `/v3/projects/54e53f317a9d47f9ad540f0a69103470/users/0d8fa1e563f541318da1bea62db2b04b/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.290690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e53f317a9d47f9ad540f0a69103470, user_id=0d8fa1e563f541318da1bea62db2b04b, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:35.293553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:35.321585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1126/2259] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:35 2022] PUT /identity/v3/projects/54e53f317a9d47f9ad540f0a69103470/users/0d8fa1e563f541318da1bea62db2b04b/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:35.323315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:35.346258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc538-3641-43e7-a374-a7d260726bd2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:35.474808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:35.475601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.475890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.476180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.477499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:35.479792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8717e-5394-436a-bb7c-91e4c349723a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:35.510628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1134/2260] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:35.514726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:35.549897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a32643-ef1a-4a98-a04d-7d1137b39e35 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:35.566132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:35.567298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.569123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.574938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] PATH_INFO: `/v3/projects/54e53f317a9d47f9ad540f0a69103470/users/0d8fa1e563f541318da1bea62db2b04b/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.580286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e53f317a9d47f9ad540f0a69103470, user_id=0d8fa1e563f541318da1bea62db2b04b, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:35.586972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:35.646702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1127/2261] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:35 2022] PUT /identity/v3/projects/54e53f317a9d47f9ad540f0a69103470/users/0d8fa1e563f541318da1bea62db2b04b/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:35.649344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:35.649642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:35.649926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:35.739463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] MFA Rules not processed for user `0d8fa1e563f541318da1bea62db2b04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:35.754152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:35.852293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:35.970693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1135/2262] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:36.214505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:36.217025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:36.413959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd96f42-c4b7-46ca-abaf-c3851b073cfa admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:36.419808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:36.420498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:36.420784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:36.421073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:36.444558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:36.446064 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:36.500836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1128/2263] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:36.641205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:36.651962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173ff4-7da4-49de-ab7a-552e9a1be2d7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:36.657189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:36.658133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:36.658793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:36.659339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:36.776516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:36.778244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1befc0c-7f49-4684-b944-13625d235ef7 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:36.797571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1136/2264] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:37.253533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:37.408042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ca74c-17da-4b90-8260-25ede8cbeebe service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:37.421478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:37.422805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:37.423096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:37.423434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] PATH_INFO: `/v3/users/b63ab3a9e56d446d95e64ed80b2ff8e1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:37.425579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63ab3a9e56d446d95e64ed80b2ff8e1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:37.428006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:37.508390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:37.538751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63ab3a9e56d446d95e64ed80b2ff8e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:37.539554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63ab3a9e56d446d95e64ed80b2ff8e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:37.558658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63ab3a9e56d446d95e64ed80b2ff8e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:37.584766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63ab3a9e56d446d95e64ed80b2ff8e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:37.596488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63ab3a9e56d446d95e64ed80b2ff8e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:37.607749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1129/2265] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:37 2022] DELETE /identity/v3/users/b63ab3a9e56d446d95e64ed80b2ff8e1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:37.613531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:37.614009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:37.614307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:37.639100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:37.641035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:37.746959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1137/2266] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:37.764459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:37.768761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:37.882969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c58e-619b-4d9f-8c0f-7dd2159dc33e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:37.897638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:37.898320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:37.898661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:37.899268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:37.920699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:37.923540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:37.961138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1130/2267] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:38.538232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:38.540337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:38.564305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a966a12-cba6-495b-a9eb-e2c8303a5312 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:38.571915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:38.583616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:38.584175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:38.585523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] PATH_INFO: `/v3/projects/347d7a0c2fb94726b94437204bb1d3b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:38.597640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347d7a0c2fb94726b94437204bb1d3b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:38.599894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:38.665087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347d7a0c2fb94726b94437204bb1d3b8', 'request_id': 'req-828a3324-c0b7-45db-b3be-b08460578922'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.672787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347d7a0c2fb94726b94437204bb1d3b8', 'request_id': 'req-828a3324-c0b7-45db-b3be-b08460578922'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.680193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347d7a0c2fb94726b94437204bb1d3b8', 'request_id': 'req-828a3324-c0b7-45db-b3be-b08460578922'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.682524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:38.701319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] The token cache is being invalidate because project 347d7a0c2fb94726b94437204bb1d3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:38.701621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.702284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.704463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1138/2268] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:38 2022] DELETE /identity/v3/projects/347d7a0c2fb94726b94437204bb1d3b8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:38.708651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:38.719640 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d04f8a-a4c9-4aea-89ff-700384c65e38 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:38.787815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:38.789136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:38.789618 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:38.790119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] PATH_INFO: `/v3/users/0d8fa1e563f541318da1bea62db2b04b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:38.791936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8fa1e563f541318da1bea62db2b04b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:38.793956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:38.867717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8fa1e563f541318da1bea62db2b04b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.869316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8fa1e563f541318da1bea62db2b04b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.882867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8fa1e563f541318da1bea62db2b04b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.884550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:38.904286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8fa1e563f541318da1bea62db2b04b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.912747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8fa1e563f541318da1bea62db2b04b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:38.919454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1131/2269] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:38 2022] DELETE /identity/v3/users/0d8fa1e563f541318da1bea62db2b04b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:26:39.069564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:39.072062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.092043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a3324-c0b7-45db-b3be-b08460578922 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:39.213340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:39.214407 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.214869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.215159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] PATH_INFO: `/v3/users/751552d2e4fe49cda44c19f0cea679cb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.217397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751552d2e4fe49cda44c19f0cea679cb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:39.219439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:39.335888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751552d2e4fe49cda44c19f0cea679cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.371082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751552d2e4fe49cda44c19f0cea679cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.397382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751552d2e4fe49cda44c19f0cea679cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.415932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:39.417685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.420790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751552d2e4fe49cda44c19f0cea679cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.422941 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-48fc71f7-c292-4747-9ea8-3808811f4167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751552d2e4fe49cda44c19f0cea679cb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.434602 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1139/2270] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:39 2022] DELETE /identity/v3/users/751552d2e4fe49cda44c19f0cea679cb => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:39.441531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe067-ce17-48d9-a7bd-8b4abc93d0f7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.441840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe067-ce17-48d9-a7bd-8b4abc93d0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.442117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fe067-ce17-48d9-a7bd-8b4abc93d0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.467500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1fce9-f596-4a90-b439-906bf2a97c1f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:39.496627 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-681fe067-ce17-48d9-a7bd-8b4abc93d0f7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:39.498128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-681fe067-ce17-48d9-a7bd-8b4abc93d0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.580500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1140/2271] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:39.582046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e5cb-f932-4f6d-8767-74fb17103ec9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.582444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e5cb-f932-4f6d-8767-74fb17103ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.582832 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5e5cb-f932-4f6d-8767-74fb17103ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.591062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:39.592114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.592406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.592689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] PATH_INFO: `/v3/projects/54e53f317a9d47f9ad540f0a69103470` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.594720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e53f317a9d47f9ad540f0a69103470)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:39.608364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:39.611064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:39.620604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d9b5e5cb-f932-4f6d-8767-74fb17103ec9 None None] MFA Rules not processed for user `247594fc7d5d446e85e1cc017a5ecea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:39.622301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d9b5e5cb-f932-4f6d-8767-74fb17103ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.677931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e53f317a9d47f9ad540f0a69103470', 'request_id': 'req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.686480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e53f317a9d47f9ad540f0a69103470', 'request_id': 'req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.694897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e53f317a9d47f9ad540f0a69103470', 'request_id': 'req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.716132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] The token cache is being invalidate because project 54e53f317a9d47f9ad540f0a69103470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:39.716804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.717337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:39.722333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1132/2272] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:39 2022] DELETE /identity/v3/projects/54e53f317a9d47f9ad540f0a69103470 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:39.723403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:39.724943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.755391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1141/2273] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:39.757054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.758845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.759749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.799480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:39.801637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.823133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ea798-9edd-4dab-a68e-4b4b7e1dc3bb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:39.827469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:39.828176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.828634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.838850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:39.841546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.864668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:39.898816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1142/2274] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:39.900248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:39.905028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:39.943756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:39.945810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f5331-1a7c-4678-8651-566fd7e2baa6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:39.976552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1133/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:39.983509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:39.984659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:39.987519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.024316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91558b-14f9-4d5b-8f05-6ff48faf8d27 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.028570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.029229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.029530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.029807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.030157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:40.032780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.036304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.056793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1134/2276] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:40.058133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.085910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bc0f8-5f9d-4c82-a291-c9213a196110 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.105695 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.106393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.106686 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.106973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.107883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:40.108172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.109869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.158099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1135/2277] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 15:26:40.160388 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.181887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645b79d-1bd0-449f-9624-b2e2cbdadcd3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.189588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.191367 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.191806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.192258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.193125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.195253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.222827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:40.268472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.270424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.278867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1136/2278] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:40 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:40.284347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.288416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1143/2279] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:40.470466 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af8990-4980-4829-b5ee-0497991001ca admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.488052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.488846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.489203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.489549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.490328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.495516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0b1b3-e398-4a99-88f9-5a69255ff974 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.537301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1137/2280] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:40 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:26:40.541124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.543878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.544449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.544884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.565860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2299af8b-3f3d-40f4-a0d6-7a3ef3814529 service glance] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.570096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:40.570096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.582369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.584059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.584448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.585028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.586472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.589000 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.626285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1144/2281] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:40.627521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.630653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.646121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1138/2282] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:40.646992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.656510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a417174-1754-423d-aa60-28e6c9155a84 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.663622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:40.670274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329f287-0a05-424b-94fd-b4923ca83ded None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.670788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.672769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.675516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.675828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] PATH_INFO: `/v3/projects/04a162567976474fb70601e678e3abb8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.677537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a162567976474fb70601e678e3abb8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.679520 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.683371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.684041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.684331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.684615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.685361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.687014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.702212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1139/2283] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:40.703202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.707345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.744826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a162567976474fb70601e678e3abb8', 'request_id': 'req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:40.752370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a162567976474fb70601e678e3abb8', 'request_id': 'req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:40.758699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a162567976474fb70601e678e3abb8', 'request_id': 'req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:40.775415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] The token cache is being invalidate because project 04a162567976474fb70601e678e3abb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:40.775801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:40.776946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:40.780018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1145/2284] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:40 2022] DELETE /identity/v3/projects/04a162567976474fb70601e678e3abb8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:40.781625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.783872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.797392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4fa72-ac10-4645-88c0-c00bcabe7baf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.900472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a59c3c-d05d-4a2a-b7f5-904352c792fe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:40.908736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.910045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.910518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.911026 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] PATH_INFO: `/v3/projects/86c15686e19f4a7e8c7f3e5b0d044700/users/f77a6a706070477c8ab45ea163c47e02/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.912790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:40.913642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:40.913913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:40.914242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:40.916408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:40.930541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c15686e19f4a7e8c7f3e5b0d044700, user_id=f77a6a706070477c8ab45ea163c47e02, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:40.934032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:40.950910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1146/2285] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:40 2022] PUT /identity/v3/projects/86c15686e19f4a7e8c7f3e5b0d044700/users/f77a6a706070477c8ab45ea163c47e02/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:40.952025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:40.958490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:41.008531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:41.010792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60d0d5-34cd-4374-a5a8-55b90932963d service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:41.019609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1140/2286] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:26:41.020916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:41.021327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:41.021582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:41.029752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:41.050946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] MFA Rules not processed for user `f77a6a706070477c8ab45ea163c47e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:41.052263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:41.058480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6c65f-dc68-4005-b500-234ae0c4608f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:41.077667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:41.078900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:41.079394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:41.079854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] PATH_INFO: `/v3/users/98055c4510dc4cbb83daab54914438cd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:41.081983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98055c4510dc4cbb83daab54914438cd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:41.084379 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:41.113916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:41.160828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98055c4510dc4cbb83daab54914438cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.164479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1141/2287] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:41.181783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98055c4510dc4cbb83daab54914438cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.182502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:41.186188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:41.192593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98055c4510dc4cbb83daab54914438cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.202429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98055c4510dc4cbb83daab54914438cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.203988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98055c4510dc4cbb83daab54914438cd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.211586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1147/2288] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:40 2022] DELETE /identity/v3/users/98055c4510dc4cbb83daab54914438cd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:41.302424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b543a14-ffba-46af-815b-5309a53698cc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:41.387279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:41.388304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:41.388486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:41.388776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:41.391546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:41.527471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:41.529228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:41.544298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1142/2289] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:26:41.646516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:41.648186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:41.753504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16ad0-29ce-411a-9f6f-10beb744939e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:41.757653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:41.758744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:41.759429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:41.760106 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] PATH_INFO: `/v3/projects/7832794269b7497fa5ced83e015cffa2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:41.775024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7832794269b7497fa5ced83e015cffa2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:41.777339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:41.855703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7832794269b7497fa5ced83e015cffa2', 'request_id': 'req-eed02731-dfae-4717-9b24-1824f41607c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.863700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7832794269b7497fa5ced83e015cffa2', 'request_id': 'req-eed02731-dfae-4717-9b24-1824f41607c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.870094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7832794269b7497fa5ced83e015cffa2', 'request_id': 'req-eed02731-dfae-4717-9b24-1824f41607c8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.874018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:41.914793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] The token cache is being invalidate because project 7832794269b7497fa5ced83e015cffa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:41.916452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.920491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:41.923270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1148/2290] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:41 2022] DELETE /identity/v3/projects/7832794269b7497fa5ced83e015cffa2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:41.941883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:41.957710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b2ba2-14d7-4f02-96d2-6c02c1f7b489 service glance] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:42.022584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:42.056011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:42.057711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:42.058273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:42.058801 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] PATH_INFO: `/v3/users/0dfc0722d35242c0b7f7ae16dac4d06e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:42.060379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dfc0722d35242c0b7f7ae16dac4d06e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:42.062606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:42.149849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dfc0722d35242c0b7f7ae16dac4d06e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:42.155116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dfc0722d35242c0b7f7ae16dac4d06e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:42.174973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dfc0722d35242c0b7f7ae16dac4d06e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:42.201424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dfc0722d35242c0b7f7ae16dac4d06e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:42.243711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ee715d-9e83-44b7-aa5a-6afb079cfbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dfc0722d35242c0b7f7ae16dac4d06e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:42.256838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1143/2291] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:26:41 2022] DELETE /identity/v3/users/0dfc0722d35242c0b7f7ae16dac4d06e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:42.934871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:42.943067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:42.970564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed02731-dfae-4717-9b24-1824f41607c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.098249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.099562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.099961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.100341 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] PATH_INFO: `/v3/projects/557c7212f01d4bfeb4b11164e881ff20` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.102052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557c7212f01d4bfeb4b11164e881ff20)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.104414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.185242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557c7212f01d4bfeb4b11164e881ff20', 'request_id': 'req-e4cfb334-64da-4820-8c67-6b58a5531c15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:43.195315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557c7212f01d4bfeb4b11164e881ff20', 'request_id': 'req-e4cfb334-64da-4820-8c67-6b58a5531c15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:43.203589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557c7212f01d4bfeb4b11164e881ff20', 'request_id': 'req-e4cfb334-64da-4820-8c67-6b58a5531c15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:43.221774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] The token cache is being invalidate because project 557c7212f01d4bfeb4b11164e881ff20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:26:43.222331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:43.223253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e4cfb334-64da-4820-8c67-6b58a5531c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:26:43.227303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1149/2292] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:26:42 2022] DELETE /identity/v3/projects/557c7212f01d4bfeb4b11164e881ff20 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:43.230715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.231287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.232111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.282248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:43.283656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:43.391270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1144/2293] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:43.398669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49712-13af-4978-8836-32c02e149401 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.399383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49712-13af-4978-8836-32c02e149401 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.399949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c49712-13af-4978-8836-32c02e149401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.438147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35c49712-13af-4978-8836-32c02e149401 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:43.439868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35c49712-13af-4978-8836-32c02e149401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:43.452534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1150/2294] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:26:43.457306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:43.479685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30319841-7dd6-464a-8afa-dc4b09813d9d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.485749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.487739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.487739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.487739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.502699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:26:43.503868 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.504828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.539392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1145/2295] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:26:43.540606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c49712-13af-4978-8836-32c02e149401 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:43.558024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c49712-13af-4978-8836-32c02e149401 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.573508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.573508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.573508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.573508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.574010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.574010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.621397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1151/2296] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:26:43 2022] POST /identity/v3/projects => generated 363 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:43.625977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:43.638005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857d9e-5955-4e3b-a103-617d35995615 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.733236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:43.775755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.779846 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.780052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.780762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.781943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.784745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.840475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1146/2297] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:26:43 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:26:43.841302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:43.860579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fb23a-0af3-4739-b960-1c13d66dca3e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.881592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.882196 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.882491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.882980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.883787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.886718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.911289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1152/2298] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:43.912841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:43.925786 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46386708-0103-4b18-be62-9e1bc132e8cc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:43.947559 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:43.953181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:43.953511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:43.954954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:43.958056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:43.965191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3134e2-b2a7-453b-8a53-be4ae515ba3f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:43.986255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1147/2299] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:26:43.989998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:44.004508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaea82f-2d2a-41e6-92e2-57a79ce08d2b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:44.012293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:44.013481 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:44.013933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:44.014376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] PATH_INFO: `/v3/projects/dda3e82a2a0947ed822cb68ce58cf6cd/users/a06ec530021e4faa8b70f87c2b947274/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:44.033667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda3e82a2a0947ed822cb68ce58cf6cd, user_id=a06ec530021e4faa8b70f87c2b947274, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:44.037391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:44.069258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1153/2300] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:26:43 2022] PUT /identity/v3/projects/dda3e82a2a0947ed822cb68ce58cf6cd/users/a06ec530021e4faa8b70f87c2b947274/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:26:44.082603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21415-21c9-4151-9832-f0cd60ee9316 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:44.083374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21415-21c9-4151-9832-f0cd60ee9316 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:44.083550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e21415-21c9-4151-9832-f0cd60ee9316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:44.132549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b0e21415-21c9-4151-9832-f0cd60ee9316 None None] MFA Rules not processed for user `a06ec530021e4faa8b70f87c2b947274`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:44.134013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b0e21415-21c9-4151-9832-f0cd60ee9316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:44.282325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1148/2301] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:44.304437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:44.307861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:44.345234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:44.395359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ff502-4aca-4ea8-8dd3-5a41979dfcf1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:44.399043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:44.399928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:44.400317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:44.400480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:44.411693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:44.413233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c98b61-8ea8-46dd-8d4e-040142806cd5 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:44.428294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1154/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:26:52.544456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:52.544971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:52.545069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:52.570596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] MFA Rules not processed for user `b57356f7b4e748438b89d69663bb99ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:52.571887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:52.589370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6073e-0c31-4e6c-8a81-416ab76e3013 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:52.589880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6073e-0c31-4e6c-8a81-416ab76e3013 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:52.590155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6073e-0c31-4e6c-8a81-416ab76e3013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:52.611357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-58a6073e-0c31-4e6c-8a81-416ab76e3013 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:26:52.612644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-58a6073e-0c31-4e6c-8a81-416ab76e3013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:52.635209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1149/2303] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:52.669339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1155/2304] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:26:52.682062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:26:52.683483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:26:52.738995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:26:52.754818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388c479-e4b9-4c5a-9035-fd4aff32d8a9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:26:52.759768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:26:52.760423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:26:52.760710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:26:52.761014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:26:52.769360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:26:52.770933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272feda-d676-4a32-81a2-9af76e7ac973 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:26:52.778717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1150/2305] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:27:02.662444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=41 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:02.775547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:09.238688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:09.238688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:09.239208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:09.308236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] MFA Rules not processed for user `68676523b99247bf8f5b9a7553e67f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:09.308236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:09.447319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1156/2306] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:19.437178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=25 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:25.573645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbad-6865-4101-9438-51b69b4ccbc6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:25.574555 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbad-6865-4101-9438-51b69b4ccbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:25.574997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20dbad-6865-4101-9438-51b69b4ccbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:25.619412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4e20dbad-6865-4101-9438-51b69b4ccbc6 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:25.620798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4e20dbad-6865-4101-9438-51b69b4ccbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:25.650957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1151/2307] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:25.664300 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:25.703116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585a9c-0320-49e4-a797-343e5a4a0cf0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:25.709068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:25.709851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:25.710148 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:25.710429 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:25.720981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:25.722790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:25.735512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1157/2308] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:27:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:27:26.946279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb863b-5918-4a0b-8d50-7b0ebc9ce458 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:26.946824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb863b-5918-4a0b-8d50-7b0ebc9ce458 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:26.946924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb863b-5918-4a0b-8d50-7b0ebc9ce458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:26.965756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-05bb863b-5918-4a0b-8d50-7b0ebc9ce458 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:26.967087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-05bb863b-5918-4a0b-8d50-7b0ebc9ce458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:26.975544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1152/2309] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:26.990780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:27.001797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05334f9-b9db-4325-a858-1928cc68acc4 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:27.006934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:27.007828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:27.008310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:27.008746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:27.019092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:27.020965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:27.030154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1158/2310] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:27:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:27:30.915115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:30.915752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:30.915872 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:30.958844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] MFA Rules not processed for user `74581e9c5b27401686b40b2312a6f2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:30.963509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:31.083408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1153/2311] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:32.018517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:32.021896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:32.218712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5ac60-c333-4b17-8a6e-ea213f6c8eaa service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:32.225330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:32.226989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:32.227419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:32.227674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] PATH_INFO: `/v3/users/857d941f699143ecbf25d842fb1f9ff8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:32.236926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=857d941f699143ecbf25d842fb1f9ff8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:32.240568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:32.343412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857d941f699143ecbf25d842fb1f9ff8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:32.368343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857d941f699143ecbf25d842fb1f9ff8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:32.381578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857d941f699143ecbf25d842fb1f9ff8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:32.392933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857d941f699143ecbf25d842fb1f9ff8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:32.394448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2621d7a8-7862-4d7a-849a-3182e3e9a415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857d941f699143ecbf25d842fb1f9ff8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:32.408690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1159/2312] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:27:32 2022] DELETE /identity/v3/users/857d941f699143ecbf25d842fb1f9ff8 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:32.408690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:32.843093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:32.861182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d34d4-9adc-462c-9dd5-da11aa065e0e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:32.977286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:32.978171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:32.978518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:32.978792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] PATH_INFO: `/v3/projects/ee14fb4cfead402ab57d78792f303b48` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:32.980127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee14fb4cfead402ab57d78792f303b48)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:32.982372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:33.045010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee14fb4cfead402ab57d78792f303b48', 'request_id': 'req-86455c74-6478-4d7c-9171-f43541f3cd08'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:33.058330 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee14fb4cfead402ab57d78792f303b48', 'request_id': 'req-86455c74-6478-4d7c-9171-f43541f3cd08'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:33.068207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee14fb4cfead402ab57d78792f303b48', 'request_id': 'req-86455c74-6478-4d7c-9171-f43541f3cd08'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:33.073482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:33.088933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] The token cache is being invalidate because project ee14fb4cfead402ab57d78792f303b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:27:33.089469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:33.090804 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:33.104454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1154/2313] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:27:32 2022] DELETE /identity/v3/projects/ee14fb4cfead402ab57d78792f303b48 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:40.710228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bba9b-226a-4859-9e36-d3c191c984d5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:40.710839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bba9b-226a-4859-9e36-d3c191c984d5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:40.711102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bba9b-226a-4859-9e36-d3c191c984d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:40.737213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a45bba9b-226a-4859-9e36-d3c191c984d5 None None] MFA Rules not processed for user `46896be59b73481287235aac3d630f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:40.738726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a45bba9b-226a-4859-9e36-d3c191c984d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:40.870873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1160/2314] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:41.605092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:41.606530 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:41.689144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86455c74-6478-4d7c-9171-f43541f3cd08 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:41.694683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:41.695626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:41.695991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:41.697146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] PATH_INFO: `/v3/users/f77a6a706070477c8ab45ea163c47e02` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:41.698984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f77a6a706070477c8ab45ea163c47e02)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:41.700971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:41.755567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f77a6a706070477c8ab45ea163c47e02'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:41.756453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f77a6a706070477c8ab45ea163c47e02'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:41.764974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f77a6a706070477c8ab45ea163c47e02'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:41.777998 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f77a6a706070477c8ab45ea163c47e02'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:41.786370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f77a6a706070477c8ab45ea163c47e02'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:41.792593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1155/2315] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:27:41 2022] DELETE /identity/v3/users/f77a6a706070477c8ab45ea163c47e02 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:41.797499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:41.797979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:41.798076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:41.825366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:41.826582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:41.889055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1161/2316] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:27:41.899154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:41.900593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:41.997125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800386-5862-4e63-9c65-a758ea7955c8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:42.002130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:42.002580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.002871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.003154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.011328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:42.017133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:42.019368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10a359-8e27-4c7d-a041-4c3e3dd56528 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:42.033186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1156/2317] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:42.352358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:42.353917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:42.369686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e23f-eefb-44ab-a786-f9e4c2a96f1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:42.375827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:42.376903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.377345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.377980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] PATH_INFO: `/v3/projects/86c15686e19f4a7e8c7f3e5b0d044700` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.379437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c15686e19f4a7e8c7f3e5b0d044700)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:42.381956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:42.436128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c15686e19f4a7e8c7f3e5b0d044700', 'request_id': 'req-af5d12e8-0755-4a2e-97de-0b7e76888953'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:42.444570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c15686e19f4a7e8c7f3e5b0d044700', 'request_id': 'req-af5d12e8-0755-4a2e-97de-0b7e76888953'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:42.462293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c15686e19f4a7e8c7f3e5b0d044700', 'request_id': 'req-af5d12e8-0755-4a2e-97de-0b7e76888953'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:42.479734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] The token cache is being invalidate because project 86c15686e19f4a7e8c7f3e5b0d044700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:27:42.479958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:42.480663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-af5d12e8-0755-4a2e-97de-0b7e76888953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:42.482249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1162/2318] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:27:42 2022] DELETE /identity/v3/projects/86c15686e19f4a7e8c7f3e5b0d044700 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:42.487060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.487385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.487673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.553488 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:42.554769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:42.618286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1157/2319] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:27:42.626721 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.627015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.627328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.648008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:42.649517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:42.662708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1163/2320] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:42.664038 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:42.674178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846211e2-4a26-48a7-8283-946dcbcee1d8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:42.679329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:42.680348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.680653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.680915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.681833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:27:42.682227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:42.683880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:42.702406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1158/2321] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:27:42.706261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:42.717052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3f5be-d842-442d-9dea-73781fbe49bc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:42.722375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:42.723635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.723943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.724223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.724935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:42.727051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:42.757554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:42.759392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:42.760322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:42.806467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1164/2322] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:27:42 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:27:42.807391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:42.955235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0bf9e-fab9-4786-8139-64aa467a2ccd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:42.963164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:42.964046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:42.964338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:42.964622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:42.965421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:42.968154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:42.996418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1165/2323] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:27:42 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:27:43.001993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.004405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.027902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1d0da-bce3-485f-b23b-785423e8fac4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.117335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:43.140560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61567efe-c6db-4002-a977-3b1cb5113912 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.142957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.143697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.143974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.144249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.146229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.146898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.147233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.147528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.148825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.149432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.166314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.168412 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.173468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.175112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.182719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1159/2324] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:43.183588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.196527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3092cb-7c31-4934-b1af-cfff8a492e36 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.198003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1166/2325] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:43.204318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.205019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.205452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.205746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.206492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.209960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.218856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.222496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1160/2326] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:27:43.227489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.227589 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.260163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d0cfc-dcd5-4610-a4d6-ed3999197340 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.263448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff9c910-31b2-4ef2-9b07-ebc6135843ac service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.271653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.273991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.274744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.275456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] PATH_INFO: `/v3/users/a06ec530021e4faa8b70f87c2b947274` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.277549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06ec530021e4faa8b70f87c2b947274)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.279780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.280590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.280851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.281562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.281921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.282896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.285434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.297858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1161/2327] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:27:43.302504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.319112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b9076-3867-4646-901c-e178c181f44a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.326636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.327658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.328128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.328424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] PATH_INFO: `/v3/projects/212d325856a04c298f71531a4339fd1c/users/2fa546a8c46c4f9186b0337403dbb43d/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.344728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=212d325856a04c298f71531a4339fd1c, user_id=2fa546a8c46c4f9186b0337403dbb43d, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.347414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46328fb5-8420-40bb-bdee-99802fa3a3ea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.348073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06ec530021e4faa8b70f87c2b947274'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.362001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06ec530021e4faa8b70f87c2b947274'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.363817 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1162/2328] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:27:43 2022] PUT /identity/v3/projects/212d325856a04c298f71531a4339fd1c/users/2fa546a8c46c4f9186b0337403dbb43d/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:27:43.372641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=5,keystone:SELECT=86,keystone:DELETE=9 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:43.374843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.375185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.375583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.380516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06ec530021e4faa8b70f87c2b947274'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.389865 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06ec530021e4faa8b70f87c2b947274'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.390806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-61e2a7f9-4213-4ae9-b1ea-6d0b4b088a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06ec530021e4faa8b70f87c2b947274'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.399860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1167/2329] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:27:43 2022] DELETE /identity/v3/users/a06ec530021e4faa8b70f87c2b947274 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:43.403953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.404678 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.404949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] MFA Rules not processed for user `2fa546a8c46c4f9186b0337403dbb43d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:43.405221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.406717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.425932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:43.427154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.482176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1163/2330] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:43.487956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1168/2331] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:27:43.494389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.495838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.497139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.498467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.554016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532df5e-3ce0-4153-a944-2cea92a3c35a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.554805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6a7a-0bf3-4025-9553-c9c803f0e5e7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.557690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.558381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.558564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.558701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.558960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.559186 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.559573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.559795 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.569499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.569659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.571018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.571125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a02cc-1557-4560-93d0-85f04e9fa47a service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.582630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1169/2333] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:27:43.583306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1164/2333] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:43.864946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:43.867664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:43.907320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99be44-06a0-436e-83d3-dbca128206e1 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:43.913412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:43.914314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:43.914629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:43.914904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] PATH_INFO: `/v3/projects/dda3e82a2a0947ed822cb68ce58cf6cd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:43.916243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda3e82a2a0947ed822cb68ce58cf6cd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:43.918262 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:43.982933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda3e82a2a0947ed822cb68ce58cf6cd', 'request_id': 'req-064938f1-baf2-431c-80ea-4f96beb02dfb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.990991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda3e82a2a0947ed822cb68ce58cf6cd', 'request_id': 'req-064938f1-baf2-431c-80ea-4f96beb02dfb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:43.998311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda3e82a2a0947ed822cb68ce58cf6cd', 'request_id': 'req-064938f1-baf2-431c-80ea-4f96beb02dfb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:44.031558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] The token cache is being invalidate because project dda3e82a2a0947ed822cb68ce58cf6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:27:44.031558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:44.031558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-064938f1-baf2-431c-80ea-4f96beb02dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:44.031558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1170/2334] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:27:43 2022] DELETE /identity/v3/projects/dda3e82a2a0947ed822cb68ce58cf6cd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:44.036053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.036509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.036655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.058747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:44.060157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:44.090475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:44.173176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1165/2335] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:44.183325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.183699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.183989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.202657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:44.203963 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:44.213105 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1171/2336] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:44.217537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.228639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8acd25-a5c3-450f-babd-10a384518da6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.234797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.236112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.236477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.236751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.237950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:27:44.238323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.240327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.261532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1166/2337] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:27:44.266243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.274536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:44.280920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43458b-26a9-48c4-8b97-67d76cfea1de None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.289803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.291494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.292342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.293431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.294311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.298239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.336122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1172/2338] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:27:44 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:27:44.337948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.348415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55e2cb-2735-40d1-b12f-d01433c05ae3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.415308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.416015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.416308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.416593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.417316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.419273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.462692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1167/2339] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:27:44 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:27:44.469889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.482856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c73ab-9b74-4400-a327-84cbfea131e2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.489242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.490286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.490561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.490836 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.491609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.493202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.509444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1173/2340] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:27:44.513625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.535881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126e93-0573-4d51-acfa-a0fe7fc63d6c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.546239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.547525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.547809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.548115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.549592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.551301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e1001-713a-45f4-be92-4f438d45bbe9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.577025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1168/2341] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:27:44.577298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.593230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5b0a-0138-4da1-a109-24a38e6f2d57 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.600810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.602110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.602271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.602529 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] PATH_INFO: `/v3/projects/82bd22b65d434a109fdf50eb6d9de43c/users/eaea801410604e1fa436e6ed973d87a5/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.623769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bd22b65d434a109fdf50eb6d9de43c, user_id=eaea801410604e1fa436e6ed973d87a5, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.627080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.648149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1174/2342] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:27:44 2022] PUT /identity/v3/projects/82bd22b65d434a109fdf50eb6d9de43c/users/eaea801410604e1fa436e6ed973d87a5/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:27:44.659169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.659496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.659776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.692098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] MFA Rules not processed for user `eaea801410604e1fa436e6ed973d87a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:44.693329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:44.777377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1169/2343] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:44.789320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:44.791301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:44.864085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab7e92-f8b7-459c-9bec-222f9af3f754 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:44.868307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:44.868967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:44.869251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:44.869527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:44.878783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:44.880236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8b509-de7c-4eef-892b-6187337ae321 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:44.889440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1175/2344] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:54.773706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:54.885015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:55.504802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:55.568072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c16e0-6335-49e0-be27-92851f7c1f37 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:55.572576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:55.573448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:55.573731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:55.574007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] PATH_INFO: `/v3/users/eaea801410604e1fa436e6ed973d87a5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:55.575190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaea801410604e1fa436e6ed973d87a5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:55.577174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:55.625550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaea801410604e1fa436e6ed973d87a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:55.626257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaea801410604e1fa436e6ed973d87a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:55.633674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaea801410604e1fa436e6ed973d87a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:55.645706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaea801410604e1fa436e6ed973d87a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:55.652474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaea801410604e1fa436e6ed973d87a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:55.658831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1170/2345] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:27:55 2022] DELETE /identity/v3/users/eaea801410604e1fa436e6ed973d87a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:55.663275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:55.663573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:55.663856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:55.681690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:27:55.682989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:55.743829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1176/2346] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:27:55.754018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:55.755450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:55.817937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca464522-96df-4f60-b0b6-fb96c7b1d9cf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:55.821613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:55.822267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:55.822553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:55.822833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:55.832094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:55.833519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:55.842853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1171/2347] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:27:56.132652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:56.136060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:56.148828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c865708-ec5e-4243-9f40-dea0c32a361d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:56.153559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:56.154688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:56.154973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:56.155273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] PATH_INFO: `/v3/projects/82bd22b65d434a109fdf50eb6d9de43c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:56.157003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] RBAC: Authorizing `identity:delete_project(project_id=82bd22b65d434a109fdf50eb6d9de43c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:56.158911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:56.206908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82bd22b65d434a109fdf50eb6d9de43c', 'request_id': 'req-ae056e93-268f-441c-a2d7-23664161472d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:56.213783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82bd22b65d434a109fdf50eb6d9de43c', 'request_id': 'req-ae056e93-268f-441c-a2d7-23664161472d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:56.220028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82bd22b65d434a109fdf50eb6d9de43c', 'request_id': 'req-ae056e93-268f-441c-a2d7-23664161472d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:56.232178 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] The token cache is being invalidate because project 82bd22b65d434a109fdf50eb6d9de43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:27:56.232486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:56.233086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:27:56.235274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1177/2348] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:27:56 2022] DELETE /identity/v3/projects/82bd22b65d434a109fdf50eb6d9de43c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:27:57.111672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:27:57.113082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:57.124150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464cca2-9116-450b-a2b2-dc6e64380556 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:27:57.142886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:27:57.177418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:27:57.178085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:27:57.178371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:27:57.178652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:27:57.180446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:27:57.241079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:27:57.242551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25635b79-dd2b-4f84-9308-e6a394b0f553 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:27:57.252399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1172/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:27:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:28:06.229183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:07.247882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=40 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:24.110034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:24.111490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.174822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae056e93-268f-441c-a2d7-23664161472d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:24.179440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:24.180311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.180598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.180879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] PATH_INFO: `/v3/users/2fa546a8c46c4f9186b0337403dbb43d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.182089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fa546a8c46c4f9186b0337403dbb43d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:24.184045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:24.229008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fa546a8c46c4f9186b0337403dbb43d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.240708 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fa546a8c46c4f9186b0337403dbb43d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.249165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fa546a8c46c4f9186b0337403dbb43d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.256814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fa546a8c46c4f9186b0337403dbb43d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.262635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fa546a8c46c4f9186b0337403dbb43d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.268929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1178/2350] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:28:24 2022] DELETE /identity/v3/users/2fa546a8c46c4f9186b0337403dbb43d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:24.274087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.274552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.274650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.292062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:24.293264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.350307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1173/2351] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:24.359891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:24.361345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.422096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc684b-5731-4328-ae91-989d31a64a73 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:24.425717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:24.426362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.427517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.427802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.436811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:24.438234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbe9b3-ba26-4235-8506-d688834d3aed service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:24.448415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1179/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:28:24.713939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:24.715546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.728076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413974-16fe-47bb-94a9-c7a26f16d056 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:24.732612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:24.733480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.733766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.734042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] PATH_INFO: `/v3/projects/212d325856a04c298f71531a4339fd1c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.735405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212d325856a04c298f71531a4339fd1c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:24.737389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:24.783440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212d325856a04c298f71531a4339fd1c', 'request_id': 'req-3d2415fd-31ae-4aa5-bfab-754816706c91'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.791153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212d325856a04c298f71531a4339fd1c', 'request_id': 'req-3d2415fd-31ae-4aa5-bfab-754816706c91'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.797295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212d325856a04c298f71531a4339fd1c', 'request_id': 'req-3d2415fd-31ae-4aa5-bfab-754816706c91'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.808086 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] The token cache is being invalidate because project 212d325856a04c298f71531a4339fd1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:28:24.808377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.809005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3d2415fd-31ae-4aa5-bfab-754816706c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:24.810526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1174/2353] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:28:24 2022] DELETE /identity/v3/projects/212d325856a04c298f71531a4339fd1c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:24.815391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.815692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.815977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.836573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:24.837794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.854757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:24.899096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1180/2354] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:24.906961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.907279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.907567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.925829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:24.927667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:24.935379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1175/2355] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:24.939289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:24.948753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfb242-ccf9-425c-9cd4-fa8d4a2a0585 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:24.953303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:24.953955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.954242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.954523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.955276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:28:24.955590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:24.957174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:24.968466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1181/2356] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:28:24.972458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:24.981977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acbfc4-fbe9-48ab-9bab-fbf51306a43c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:24.986684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:24.987366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:24.987654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:24.987932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:24.988625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:24.991156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.015726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1176/2357] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:28:24 2022] POST /identity/v3/projects => generated 397 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:28:25.019822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:25.030034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179ace8-fee1-4fde-9cf6-a032b45318c9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:25.085312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:25.085910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.086193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.086469 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.087159 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:25.089116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.108606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1182/2358] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:28:25 2022] POST /identity/v3/users => generated 299 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:28:25.112864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:25.123385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9e279-8fd2-4832-823d-9aabb11ba122 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:25.127977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:25.128642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.128926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.129198 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.130003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:25.131773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.142118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1177/2359] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:28:25.147050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:25.157917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb9f9-ac38-47e4-b208-d9cc53ce37ee admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:25.163087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:25.163780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.164066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.164345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.165077 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:25.166656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9fd5e-bbe9-4378-8d31-74ef3d74ff0d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.176921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1183/2360] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:28:25.180969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:25.190620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae2bea-0273-44f8-a4d7-dd695c0e862b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:25.195154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:25.196054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.196341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.196621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] PATH_INFO: `/v3/projects/00a3e9ad6c93493faf7f0372d1888813/users/72331e1415ac41bf800b57e8a76c1ca9/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.211148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a3e9ad6c93493faf7f0372d1888813, user_id=72331e1415ac41bf800b57e8a76c1ca9, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:25.213794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.227137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1178/2361] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:28:25 2022] PUT /identity/v3/projects/00a3e9ad6c93493faf7f0372d1888813/users/72331e1415ac41bf800b57e8a76c1ca9/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:25.237161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.237458 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.237767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.263988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] MFA Rules not processed for user `72331e1415ac41bf800b57e8a76c1ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:25.265216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:25.327993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1184/2362] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:25.339646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:25.341254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:25.364975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:25.421351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e97f3-1a2e-47e7-88ab-ece234d3b754 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:25.424977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:25.425626 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:25.425914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:25.426194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:25.434341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:25.435818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd687-0d01-4601-81a8-ea5518da57c9 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:25.444513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1179/2363] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:28:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:28:34.477914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:34.497011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:34.540089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7f736-7a69-4e4e-871d-8faf964196b0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:34.544393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:34.545292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:34.545573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:34.545848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] PATH_INFO: `/v3/users/72331e1415ac41bf800b57e8a76c1ca9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:34.547022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72331e1415ac41bf800b57e8a76c1ca9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:34.548968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:34.593345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72331e1415ac41bf800b57e8a76c1ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:34.605559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72331e1415ac41bf800b57e8a76c1ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:34.612765 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72331e1415ac41bf800b57e8a76c1ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:34.620144 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72331e1415ac41bf800b57e8a76c1ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:34.620769 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72331e1415ac41bf800b57e8a76c1ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:34.626486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1185/2364] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:28:34 2022] DELETE /identity/v3/users/72331e1415ac41bf800b57e8a76c1ca9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:34.630938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:34.631410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:34.631647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:34.650670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:34.651890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:34.709467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1180/2365] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:34.718833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:34.720308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:34.783100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1fc50-b1dc-4f99-94da-17a3e874a7c1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:34.787275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:34.787939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:34.788224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:34.788503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:34.797883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:34.799273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:34.809595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1186/2366] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:28:35.303501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:35.305712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:35.324067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cbe7e7-82c1-49d7-8bce-423345c66716 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:35.329785 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:35.330774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:35.331244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:35.331543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] PATH_INFO: `/v3/projects/00a3e9ad6c93493faf7f0372d1888813` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:35.333646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a3e9ad6c93493faf7f0372d1888813)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:35.336123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:35.386670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a3e9ad6c93493faf7f0372d1888813', 'request_id': 'req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:35.394277 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a3e9ad6c93493faf7f0372d1888813', 'request_id': 'req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:35.401312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a3e9ad6c93493faf7f0372d1888813', 'request_id': 'req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:35.414883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] The token cache is being invalidate because project 00a3e9ad6c93493faf7f0372d1888813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:28:35.415422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:35.416160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:35.418053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1181/2367] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:28:35 2022] DELETE /identity/v3/projects/00a3e9ad6c93493faf7f0372d1888813 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:44.805072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:45.411031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:58.955008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:58.956573 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:58.970542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7de02-a15c-4b6e-a349-2355fc1b8c67 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.029946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.030829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.031113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.031445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] PATH_INFO: `/v3/users/74581e9c5b27401686b40b2312a6f2fb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.032636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74581e9c5b27401686b40b2312a6f2fb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.034584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.083472 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74581e9c5b27401686b40b2312a6f2fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.095532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74581e9c5b27401686b40b2312a6f2fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.103684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74581e9c5b27401686b40b2312a6f2fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.110812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74581e9c5b27401686b40b2312a6f2fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.111509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-068e56e8-6529-4d7d-9f28-c2329aa66edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74581e9c5b27401686b40b2312a6f2fb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.118614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1187/2368] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:28:58 2022] DELETE /identity/v3/users/74581e9c5b27401686b40b2312a6f2fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:28:59.290818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.292307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:59.315383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4c31-6e57-4585-a7e0-8e5fffd00c9d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.395677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.396564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.397227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.397511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] PATH_INFO: `/v3/projects/a75a2d018dff40bfb9b5bd72a678826c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.399206 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75a2d018dff40bfb9b5bd72a678826c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.401134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.451878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75a2d018dff40bfb9b5bd72a678826c', 'request_id': 'req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.461240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75a2d018dff40bfb9b5bd72a678826c', 'request_id': 'req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.469002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75a2d018dff40bfb9b5bd72a678826c', 'request_id': 'req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.484354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] The token cache is being invalidate because project a75a2d018dff40bfb9b5bd72a678826c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:28:59.484666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.485516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6a1fafa6-2f7a-4c8b-8712-ec344978bf81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:28:59.487384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1182/2369] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:28:59 2022] DELETE /identity/v3/projects/a75a2d018dff40bfb9b5bd72a678826c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:28:59.494985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.495333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.495575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.518032 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:59.519480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:59.592011 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1188/2370] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:28:59.600045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575449-7497-4281-8eec-36ec77038812 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.600341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575449-7497-4281-8eec-36ec77038812 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.600696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14575449-7497-4281-8eec-36ec77038812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.622322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-14575449-7497-4281-8eec-36ec77038812 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:28:59.623757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-14575449-7497-4281-8eec-36ec77038812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:28:59.632881 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1183/2371] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:28:59.637391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.648883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9541da-e0b7-40fa-8266-52bfbf8839ac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.655357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.656152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.656674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.657720 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.658120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:28:59.658505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.660699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.681131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1189/2372] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:28:59.685537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14575449-7497-4281-8eec-36ec77038812 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.695128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14575449-7497-4281-8eec-36ec77038812 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.699875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.700557 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.700845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.701127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.701838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.703818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.732592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1184/2373] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:28:59 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:28:59.737571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.748398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a61bb-b700-4e20-8c1d-6c52ba4fced5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.752045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:28:59.817527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.818218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.818647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.818928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.819683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.821710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.845169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1190/2374] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:28:59 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:28:59.849396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.860347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b87217-668f-4d0b-b0d3-c47dc80bf6c6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.865167 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.865840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.866125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.866403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.867774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.869371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.880419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1185/2375] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:28:59.884288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.896742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669e141-9066-46e0-814b-fb7cd0a14d81 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.901699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.902890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.903270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.903628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.904444 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.906096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2ca5-629b-4649-bd7a-aa8e54193264 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.920922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1191/2376] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:28:59.922054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:28:59.933510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba9b06-1aca-46d8-af05-a0b02abb677f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:28:59.938538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:28:59.939506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.939816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.940115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] PATH_INFO: `/v3/projects/42279c5b622c47a192c529ce53566555/users/f8ae63f0fa9e4c9ab82d46d6d24d22e3/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.955874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42279c5b622c47a192c529ce53566555, user_id=f8ae63f0fa9e4c9ab82d46d6d24d22e3, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:28:59.958679 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:28:59.975056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1186/2377] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:28:59 2022] PUT /identity/v3/projects/42279c5b622c47a192c529ce53566555/users/f8ae63f0fa9e4c9ab82d46d6d24d22e3/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:28:59.980969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b90a0-e0dd-402a-84b0-24a7160dd194 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:28:59.980969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b90a0-e0dd-402a-84b0-24a7160dd194 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:28:59.981565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b90a0-e0dd-402a-84b0-24a7160dd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:28:59.999547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5b6b90a0-e0dd-402a-84b0-24a7160dd194 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:00.000766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b6b90a0-e0dd-402a-84b0-24a7160dd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:00.062408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1192/2378] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:00.072394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:00.074177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:00.107336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:00.136140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d77a46-350d-4120-968f-bbf7ab3b066e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:00.139823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:00.140504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:00.140796 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:00.141076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:00.150730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:00.152250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:00.162622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1187/2379] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:05.928512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:05.929513 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:05.929610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:05.963007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] MFA Rules not processed for user `f8ae63f0fa9e4c9ab82d46d6d24d22e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:05.964319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:06.031603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1193/2380] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:06.037887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.049747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92298e98-462e-4cd0-964b-166f7ef2b058 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.055014 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.055720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.056010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.056291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.056997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.058925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.091279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1188/2381] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:29:06 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:29:06.095057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.106633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f546110-bc65-4f5b-954e-0637de2e8c7f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.131622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:06.198598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.199526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.199886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.200234 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.201100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.203375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.236551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1194/2382] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:29:06 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:29:06.240659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.253985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08964451-47a9-463f-ae02-486f81a4c571 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.260782 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.261574 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.261962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.262259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.267439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.267439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.278598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1189/2383] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:29:06.283278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.295779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136a8c-f9fe-4b84-99d6-7f066bd62e55 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.300655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.301580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.302118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.302118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] PATH_INFO: `/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e/users/1b49636b617b40708242bf30cf7db4a1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.322495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f8b24a40b54ec78fb9180d2f2ff24e, user_id=1b49636b617b40708242bf30cf7db4a1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.326273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.352911 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1195/2384] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:29:06 2022] PUT /identity/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e/users/1b49636b617b40708242bf30cf7db4a1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:29:06.357214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.369500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1663-620c-4476-bfa5-309e22c1d476 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.464521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.465481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.466020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.466538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.467406 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.469133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326d5fe-fff5-42e9-b2f2-6eca2b9c9d14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.484359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1190/2385] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:29:06.486256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:06.504131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab30b3a-e879-4cd5-b66b-850fc36e6512 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:06.514228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:06.515289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:06.516002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:06.516002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] PATH_INFO: `/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e/users/1b49636b617b40708242bf30cf7db4a1/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:06.518280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f8b24a40b54ec78fb9180d2f2ff24e, user_id=1b49636b617b40708242bf30cf7db4a1, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:06.521514 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:06.539432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1196/2386] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:29:06 2022] PUT /identity/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e/users/1b49636b617b40708242bf30cf7db4a1/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:29:13.005925 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025afa52-8840-4523-a682-4bd392097220 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:13.006444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025afa52-8840-4523-a682-4bd392097220 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:13.006842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025afa52-8840-4523-a682-4bd392097220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:13.037271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-025afa52-8840-4523-a682-4bd392097220 None None] MFA Rules not processed for user `1b49636b617b40708242bf30cf7db4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:13.039344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-025afa52-8840-4523-a682-4bd392097220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:13.140844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1191/2387] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:13.156347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:13.158923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:13.189810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa505e73-f567-4588-b8ce-f809a626ed47 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:13.190193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa505e73-f567-4588-b8ce-f809a626ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:13.190543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa505e73-f567-4588-b8ce-f809a626ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:13.209656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aa505e73-f567-4588-b8ce-f809a626ed47 None None] MFA Rules not processed for user `68676523b99247bf8f5b9a7553e67f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:13.211613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aa505e73-f567-4588-b8ce-f809a626ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:13.225139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:13.277411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1192/2388] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:29:13.282334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b14135-3dab-4c2f-9da2-e2864852b938 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:13.289622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:13.290465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:13.290584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:13.290850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:13.333321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:13.354120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:13.355763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f557a5d-6721-472c-8dce-279fc8b71929 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:13.364679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1197/2389] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:14.637320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:14.637933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:14.638195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:14.659108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] MFA Rules not processed for user `f8ae63f0fa9e4c9ab82d46d6d24d22e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:14.660903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:14.669683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1193/2390] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:19.096572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:19.097593 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:19.098130 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:19.117107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] MFA Rules not processed for user `46896be59b73481287235aac3d630f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:19.118328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:19.177604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1198/2391] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:24.652570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=20 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:29.170689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=35 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:36.597726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:36.651323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f8f03-25c6-4ca6-b5a5-47c844ca89ae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:36.655088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:36.655762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:36.656055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:36.656334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:36.658161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:36.671714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:36.673134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:36.683876 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1194/2392] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:37.850265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:37.851830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:37.863491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3cbe4-b0f3-4e5d-a351-ff7c217e2710 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:37.866969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:37.868012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:37.868559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:37.868840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:37.870527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:37.883716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:37.885125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:37.895709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1199/2393] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:46.566398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:46.578140 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f324f9-f103-4976-9b64-6a6180c71b7d service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:46.583999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:46.584478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:46.584765 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:46.585047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:46.587040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:46.644484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:46.646069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7145a-d25d-4d99-a87b-4605c763d2e3 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:46.655360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1195/2394] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:47.891028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:29:49.915857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:49.924897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635e28d-214b-4cc4-bbca-e8fb0188716e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:49.939318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:49.940004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:49.940293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:49.940575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:49.950505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:49.951965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:49.962891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1200/2395] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:29:52.288699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:52.289251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:52.289251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:52.307502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] MFA Rules not processed for user `0f5497ff570f4affbbd1b2f9129c24ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:29:52.308847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:52.316059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1196/2396] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:29:52.626113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:29:52.636648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fff7-1d6e-419c-a920-a113c199a7c7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:29:52.640340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:29:52.640991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:29:52.641279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:29:52.641561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:29:52.643336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:29:52.653415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:29:52.655092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:29:52.663910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1201/2397] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:01.469288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:01.479374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338579dc-f02c-454d-b2bd-f8a6fe750348 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:01.482987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:01.483667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:01.483987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:01.484257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:01.485999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:01.495233 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:01.496668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:01.504337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1197/2398] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:02.660728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=12 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:06.083834 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:06.085736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:06.140164 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb375f9-3110-4caf-8433-0611a63f5c3b service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:06.143746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:06.144391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:06.144677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:06.144957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:06.146683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:06.157924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:06.159380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:06.168925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1202/2399] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:11.500954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=63 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:12.159435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:12.177215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f197b-711e-41bb-a763-155cdd57e84b service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:12.184955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:12.185623 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:12.185989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:12.186257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:12.188145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:12.201520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:12.203684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:12.216778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1198/2400] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:12.891498 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:12.905255 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72213dd-6c97-43aa-8a7c-203f102cbf92 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:12.910926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:12.911631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:12.911926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:12.912213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:12.914033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:12.925274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:12.926900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af56c8a-ace7-4e66-9167-cb9bd439ae28 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:12.937335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1203/2401] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:22.211627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:22.932868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=32 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:30.753142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:30.762675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1644c-21b4-4634-b618-ea2ab1932a3f service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:30.766371 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:30.767018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:30.767347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:30.767627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:30.769433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:30.834757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:30.836282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:30.846326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1199/2402] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:31.141171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfadffa-d515-444e-b40b-cd5f556dcb6c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:31.141693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfadffa-d515-444e-b40b-cd5f556dcb6c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:31.141786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfadffa-d515-444e-b40b-cd5f556dcb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:31.220763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4cfadffa-d515-444e-b40b-cd5f556dcb6c None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:31.222273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cfadffa-d515-444e-b40b-cd5f556dcb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:31.306511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1204/2403] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:30:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:30:31.436458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:31.437999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:31.554617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68101cb2-eb33-482f-a60a-b61d44a4bec5 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:31.559819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:31.560478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:31.560763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:31.561037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:31.570191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:31.572570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:31.586608 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1200/2404] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:30:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:30:34.000537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:34.001017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:34.001132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:34.019997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] MFA Rules not processed for user `68676523b99247bf8f5b9a7553e67f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:34.021426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:34.028895 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1205/2405] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:30:34.424446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:34.437880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8b389-61b5-4489-b241-9cda1f112948 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:34.443157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:34.444445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:34.444918 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:34.445392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:34.447802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:34.466262 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:34.467767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:34.477807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1201/2406] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:34.821688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:34.823803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:34.896575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3509f-aaca-4a36-a2c1-acc71d80034d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:34.906693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:34.907674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:34.908035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:34.908416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:34.910288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:34.924091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:34.925683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:34.936237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1206/2407] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:35.825453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:35.843374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7f79a-da00-4070-806e-47f754b7227c service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:35.874068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:35.874808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:35.875098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:35.875411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:35.878755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:35.918606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:35.938517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:35.943958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:35.944434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:35.963405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae9cb0-b3ce-4ecc-9dc1-61f58d45c632 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:35.969978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:35.971113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:35.972414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1202/2408] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:35.982738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:35.983649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] PATH_INFO: `/v3/users/46896be59b73481287235aac3d630f4e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:35.987984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46896be59b73481287235aac3d630f4e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:35.991936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:36.101214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:36.109068 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46896be59b73481287235aac3d630f4e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.157203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46896be59b73481287235aac3d630f4e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.186595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46896be59b73481287235aac3d630f4e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.196987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46896be59b73481287235aac3d630f4e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.197633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f0a891e4-ab38-4524-b8a7-1df4a5acece5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46896be59b73481287235aac3d630f4e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.209181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1207/2409] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:30:35 2022] DELETE /identity/v3/users/46896be59b73481287235aac3d630f4e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:36.643593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:36.660818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b2b38-b10d-4cc6-b98e-0d2611595cab service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:36.815051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:36.880019 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:36.881921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:36.882216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:36.883656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] PATH_INFO: `/v3/projects/ef44ecc951964d51925463c5e09f9fbb` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:36.885973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef44ecc951964d51925463c5e09f9fbb)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:36.888575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:36.978971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef44ecc951964d51925463c5e09f9fbb', 'request_id': 'req-b5ec404f-59a3-42ca-a6fe-6735597be68a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:36.996107 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef44ecc951964d51925463c5e09f9fbb', 'request_id': 'req-b5ec404f-59a3-42ca-a6fe-6735597be68a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:37.007495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef44ecc951964d51925463c5e09f9fbb', 'request_id': 'req-b5ec404f-59a3-42ca-a6fe-6735597be68a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:37.031386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] The token cache is being invalidate because project ef44ecc951964d51925463c5e09f9fbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:30:37.031948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:37.032843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b5ec404f-59a3-42ca-a6fe-6735597be68a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:37.035545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1203/2410] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:30:36 2022] DELETE /identity/v3/projects/ef44ecc951964d51925463c5e09f9fbb => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:37.040758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.041245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.041586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.069590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:37.071206 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:37.266357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1208/2411] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:30:37.274649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.275597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.276089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.306222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:37.307932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:37.318116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1204/2412] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:30:37.322278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.336190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73c104-6ae2-4765-b9b6-3c5c7773de94 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.344709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.345612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.345988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.346367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.347350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:30:37.347694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.354776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.377387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1209/2413] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:30:37.385657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.405240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea0473-e108-4003-9de1-5d948a29375b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.416694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.419983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.420370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.420902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.424386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.426587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.482179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1205/2414] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:30:37 2022] POST /identity/v3/projects => generated 389 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:30:37.486658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.513978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a816d-6db8-41a6-97bf-45377049ab5d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.605004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.605703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.606046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.606470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.607385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.610447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.638788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1210/2415] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:30:37 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:30:37.642545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.654565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523f89d-b406-4a84-9705-a90397367eaf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.659600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.660280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.660569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.660847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.661620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.663318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.678040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1206/2416] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:30:37.682671 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.696373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e3a97-ca0b-4707-88cd-b24bf58377f6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.701707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.702588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.703246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.703692 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.704436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.706070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636d3a4-bcae-4b4d-8aec-fc8a57319d00 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.721545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1211/2417] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:30:37.725900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.736620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5896a2-e26c-4dd0-b390-15560bea464f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:37.742531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:37.742960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.743282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.743566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] PATH_INFO: `/v3/projects/848d6c1f478d4dba813663c4c866d8b8/users/2f35de73cde9436c952e0d5f95e74b8b/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.764702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848d6c1f478d4dba813663c4c866d8b8, user_id=2f35de73cde9436c952e0d5f95e74b8b, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:37.767444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:37.784283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1207/2418] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:30:37 2022] PUT /identity/v3/projects/848d6c1f478d4dba813663c4c866d8b8/users/2f35de73cde9436c952e0d5f95e74b8b/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:37.789314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210b05-0ff0-4467-9074-801b6142aaa7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:37.789617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210b05-0ff0-4467-9074-801b6142aaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:37.789917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210b05-0ff0-4467-9074-801b6142aaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:37.825184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a3210b05-0ff0-4467-9074-801b6142aaa7 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:37.826422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a3210b05-0ff0-4467-9074-801b6142aaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:37.851841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:37.901744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1212/2419] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:30:37.912505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:37.915274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:38.001144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f053821-d8f5-48af-87af-e080e8f196c4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:38.007989 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:38.008898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:38.009344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:38.009942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:38.024376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:38.027202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:38.041720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1208/2420] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:44.297069 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:44.297579 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:44.297661 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:44.352914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] MFA Rules not processed for user `2f35de73cde9436c952e0d5f95e74b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:44.362939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:44.481621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1213/2421] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:30:44.509440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:44.512611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:44.529323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:44.713366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58883fd-207b-4e9e-823f-088145eb3d1a service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:44.731534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:44.731534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:44.732027 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:44.733236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:44.772938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:44.776694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:44.798711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1209/2422] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:30:45.651394 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:45.653746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:45.675762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:45.695100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3af26-45fa-49c0-87a7-b665b0bb7e80 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:45.696938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7260-9aba-4a92-a394-4f781706741e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:45.707699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:45.709657 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:45.710406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:45.713544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] PATH_INFO: `/v3/users/b57356f7b4e748438b89d69663bb99ee` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:45.714036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:45.715591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:45.717041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:45.717363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57356f7b4e748438b89d69663bb99ee)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:45.717847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:45.720065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:45.720650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:45.759059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:45.761035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:45.781396 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1210/2423] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:30:45.834699 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57356f7b4e748438b89d69663bb99ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:45.857320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57356f7b4e748438b89d69663bb99ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:45.873345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57356f7b4e748438b89d69663bb99ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:45.885074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57356f7b4e748438b89d69663bb99ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:45.885881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57356f7b4e748438b89d69663bb99ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:45.898539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1214/2424] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:30:45 2022] DELETE /identity/v3/users/b57356f7b4e748438b89d69663bb99ee => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:46.297657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:46.328444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4243-2007-44d2-8aac-1b656a891113 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:46.495113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:46.497082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:46.497598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:46.498039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] PATH_INFO: `/v3/projects/0805ca8c35a34d0dbf0f1e7ac459c517` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:46.499665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0805ca8c35a34d0dbf0f1e7ac459c517)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:46.501924 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:46.607966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0805ca8c35a34d0dbf0f1e7ac459c517', 'request_id': 'req-a7c43842-5800-4550-939f-3d0573615f88'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.617234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0805ca8c35a34d0dbf0f1e7ac459c517', 'request_id': 'req-a7c43842-5800-4550-939f-3d0573615f88'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.635317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0805ca8c35a34d0dbf0f1e7ac459c517', 'request_id': 'req-a7c43842-5800-4550-939f-3d0573615f88'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.647811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] The token cache is being invalidate because project 0805ca8c35a34d0dbf0f1e7ac459c517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:30:46.648497 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.649385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a7c43842-5800-4550-939f-3d0573615f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.651994 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1211/2425] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:30:46 2022] DELETE /identity/v3/projects/0805ca8c35a34d0dbf0f1e7ac459c517 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:46.655681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:46.657180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:46.707822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506dbd7-8092-4dae-9dc5-06d3b2c2efb0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:46.759463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:46.839784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:46.841501 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:46.841935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:46.842447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] PATH_INFO: `/v3/users/247594fc7d5d446e85e1cc017a5ecea1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:46.844190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=247594fc7d5d446e85e1cc017a5ecea1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:46.846660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:46.922809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247594fc7d5d446e85e1cc017a5ecea1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.942997 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247594fc7d5d446e85e1cc017a5ecea1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.955507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247594fc7d5d446e85e1cc017a5ecea1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.978301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247594fc7d5d446e85e1cc017a5ecea1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.979668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247594fc7d5d446e85e1cc017a5ecea1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:46.989713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1215/2426] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:30:46 2022] DELETE /identity/v3/users/247594fc7d5d446e85e1cc017a5ecea1 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:47.010715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:47.011325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:47.011325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:47.063554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] MFA Rules not processed for user `2f35de73cde9436c952e0d5f95e74b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:30:47.065016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:47.076302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:47.206482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1212/2427] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:30:47.726161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:30:47.728750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:30:47.806737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850e0ad-154d-4d72-8c88-a44c70d5ad1e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:30:47.812114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:30:47.818982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:30:47.819669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:30:47.820308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] PATH_INFO: `/v3/projects/a06421484e18450fa03768d5949a0893` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:30:47.822131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06421484e18450fa03768d5949a0893)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:30:47.828567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:30:47.959992 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:47.963850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06421484e18450fa03768d5949a0893', 'request_id': 'req-64589523-37aa-435c-ac2a-1cc47ab18b72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:47.987515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06421484e18450fa03768d5949a0893', 'request_id': 'req-64589523-37aa-435c-ac2a-1cc47ab18b72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:48.004874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06421484e18450fa03768d5949a0893', 'request_id': 'req-64589523-37aa-435c-ac2a-1cc47ab18b72'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:48.040757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] The token cache is being invalidate because project a06421484e18450fa03768d5949a0893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:30:48.041316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:48.043424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:30:48.047354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1216/2428] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:30:47 2022] DELETE /identity/v3/projects/a06421484e18450fa03768d5949a0893 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:30:57.194337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=21 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:30:58.034728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:00.561519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:00.562950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:00.637296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f513611-47f1-477d-b1f8-480b593da08d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:00.700678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:00.701261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:00.701556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:00.701830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:00.703813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:00.720745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:00.722552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:00.733833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1213/2429] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:10.729176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=62 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:17.890831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:17.892280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:17.949545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64589523-37aa-435c-ac2a-1cc47ab18b72 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:17.953145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:17.953798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:17.954083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:17.954363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:17.956143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:18.018985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:18.020459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:18.030987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1217/2430] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:21.082154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.084384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:21.137730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.139546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:21.155712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03465413-940b-446f-9537-ccc20833fb4e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.160554 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.161574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.161926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.162274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] PATH_INFO: `/v3/users/0f5497ff570f4affbbd1b2f9129c24ac` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.163607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5497ff570f4affbbd1b2f9129c24ac)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.165585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.172080 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869a212-d03a-4368-9670-484b222f45e2 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.177149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.177808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.178094 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.178375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.180194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:21.193280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.194933 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.207515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1214/2431] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:21.215369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5497ff570f4affbbd1b2f9129c24ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.232263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5497ff570f4affbbd1b2f9129c24ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.244464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5497ff570f4affbbd1b2f9129c24ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.252556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5497ff570f4affbbd1b2f9129c24ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.253222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ae811ea5-6703-45bb-b2fb-983fcb23985c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5497ff570f4affbbd1b2f9129c24ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.259706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1218/2432] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:21 2022] DELETE /identity/v3/users/0f5497ff570f4affbbd1b2f9129c24ac => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:21.432441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.443875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95771-a44b-402c-80a1-0ce1423b814c service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.500254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.501120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.501398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.501670 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] PATH_INFO: `/v3/projects/6707399c704e4c97939dc84df58cb39f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.502824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6707399c704e4c97939dc84df58cb39f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.504825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.548932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6707399c704e4c97939dc84df58cb39f', 'request_id': 'req-8dd90ebf-71b3-442d-b347-840470a63ceb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.556370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6707399c704e4c97939dc84df58cb39f', 'request_id': 'req-8dd90ebf-71b3-442d-b347-840470a63ceb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.562849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6707399c704e4c97939dc84df58cb39f', 'request_id': 'req-8dd90ebf-71b3-442d-b347-840470a63ceb'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.573733 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] The token cache is being invalidate because project 6707399c704e4c97939dc84df58cb39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:21.574023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.574652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8dd90ebf-71b3-442d-b347-840470a63ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:21.576249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1215/2433] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:21 2022] DELETE /identity/v3/projects/6707399c704e4c97939dc84df58cb39f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:21.581552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.581916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.582285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.601181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:21.602383 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:21.619285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:21.664640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1219/2434] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:21.672348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.672646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.672935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.691194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:21.692376 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:21.700748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1216/2435] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:21.704722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.715139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a7ef58-d111-4a22-9569-566a883a1d33 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.719685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.720343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.720782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.721223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.721995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:21.722303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.723951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.735533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1220/2436] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:21.739904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.749700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee3793-e282-45d1-9a4e-87e750cd87db None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.754404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.755059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.756077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.756366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.757087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.758985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.769079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:21.796416 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1217/2437] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:21 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:21.799917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.809776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0d174-ecff-4d82-956d-ef7b83ad42c4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.864867 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.865542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.865831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.866111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.866796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.868740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.887566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1221/2438] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:21 2022] POST /identity/v3/users => generated 286 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:21.891614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.902622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919bdf0-73fd-4d43-8bba-a4b4c29587dc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.907199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.907889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.908176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.908456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.909188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.910830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.920684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1218/2439] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:21.924374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.934331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d1c65-8e2e-4a53-830f-7c2b4e6fbb14 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.938938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.939641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.939960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.940210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.940957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.943331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970af11-fb54-4e4f-88ef-2ba2790e4908 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:21.954100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1222/2440] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:21.959060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:21.970183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68887bf-634e-4878-85f1-cb81dbd4aae2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:21.975208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:21.976136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:21.976428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:21.976709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] PATH_INFO: `/v3/projects/b0d76cc3eba54cca8e3ffee467e4487d/users/f1c93f4c328f4b0baf9476b1c18848df/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:21.991527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d76cc3eba54cca8e3ffee467e4487d, user_id=f1c93f4c328f4b0baf9476b1c18848df, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:21.994153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:22.008120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1219/2441] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:21 2022] PUT /identity/v3/projects/b0d76cc3eba54cca8e3ffee467e4487d/users/f1c93f4c328f4b0baf9476b1c18848df/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:22.018330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:22.018632 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:22.018921 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:22.045537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] MFA Rules not processed for user `f1c93f4c328f4b0baf9476b1c18848df`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:22.046822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:22.113368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1223/2442] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:22.124773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:22.126533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:22.206767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dd6a5-dda4-44d7-bfa1-8339eb865375 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:22.212246 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:22.212895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:22.213183 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:22.213463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:22.220975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:22.222553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9159468-3d7e-48d8-97b1-fe85667ec1d6 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:22.231119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1220/2443] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:24.910208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:24.926548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:24.971170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c40b3-708d-47f2-8349-2fc71d024c11 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:24.975789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:24.976663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:24.976947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:24.977224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] PATH_INFO: `/v3/users/f1c93f4c328f4b0baf9476b1c18848df` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:24.978396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1c93f4c328f4b0baf9476b1c18848df)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:24.980349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.024978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1c93f4c328f4b0baf9476b1c18848df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.036494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1c93f4c328f4b0baf9476b1c18848df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.043663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1c93f4c328f4b0baf9476b1c18848df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.050485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1c93f4c328f4b0baf9476b1c18848df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.051135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1c93f4c328f4b0baf9476b1c18848df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.057523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1224/2444] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:24 2022] DELETE /identity/v3/users/f1c93f4c328f4b0baf9476b1c18848df => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:25.062058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.062532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.062633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.080221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:25.081408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:25.139585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1221/2445] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:25.149219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.150636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:25.206382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c411d94-c1e9-4e0c-aedb-3b74d128b325 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.209951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.210624 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.210909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.211185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.220087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.221511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de9051-94f1-44af-8562-22f98dd07b0e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.230672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1225/2446] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:31:25.508722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.510265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:25.522545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157d951-7ae3-414c-aaf7-61a17f4c176a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.527537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.528422 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.528711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.528992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] PATH_INFO: `/v3/projects/b0d76cc3eba54cca8e3ffee467e4487d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.530201 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d76cc3eba54cca8e3ffee467e4487d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.532184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.565428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:25.581837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d76cc3eba54cca8e3ffee467e4487d', 'request_id': 'req-c95d5c9a-37a3-405a-8c20-4ac775621133'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.588740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d76cc3eba54cca8e3ffee467e4487d', 'request_id': 'req-c95d5c9a-37a3-405a-8c20-4ac775621133'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.594835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d76cc3eba54cca8e3ffee467e4487d', 'request_id': 'req-c95d5c9a-37a3-405a-8c20-4ac775621133'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.605518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] The token cache is being invalidate because project b0d76cc3eba54cca8e3ffee467e4487d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:25.605810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.606438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c95d5c9a-37a3-405a-8c20-4ac775621133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:25.607996 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1222/2447] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:25 2022] DELETE /identity/v3/projects/b0d76cc3eba54cca8e3ffee467e4487d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:25.613013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.613305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.613610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.631973 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:25.633156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:25.665989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:25.692998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1226/2448] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:25.701531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.701827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.702131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.720165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:25.721374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:25.729870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1223/2449] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:25.734125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.744321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2f09-cf4c-4b4f-bfe9-f7b7fc28cef2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.748828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.749482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.749768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.750046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.750778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:25.751116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.752737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.764070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1227/2450] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:25.768098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.778301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c90f5-726e-41b2-91c3-f6fb1dc2f0e5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.782885 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.783723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.784194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.784508 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.785204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.787720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.812540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1224/2451] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:25 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:25.816683 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.826750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c78e31-f1bf-4cce-ab66-53164a67bd4e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.884072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.884747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.885039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.885323 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.886023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.888007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.908307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1228/2452] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:25 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:25.912842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.923061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8b5d7-48a1-474b-8969-f0fe132fe2f5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.927673 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.928341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.928631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.928913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.929644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.931255 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.941408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1225/2453] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:25.945134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.955317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6dfa-bb8c-4b15-a4ee-1f56e0305ece admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.960431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.961089 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.961376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.961655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:25.962386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:25.964007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6abdb-cfbc-4f3d-ba3b-f04d2a0eaff6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:25.973687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1229/2454] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:25.978158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:25.988012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3267293-f6fd-47fb-9ff3-ff6c4ab7ff05 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:25.992833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:25.993717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:25.994111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:25.994397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] PATH_INFO: `/v3/projects/c56b20a8390c487db3747378dcd79259/users/f4c2224dbf1e49a3b3bd2051da895a31/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.008859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56b20a8390c487db3747378dcd79259, user_id=f4c2224dbf1e49a3b3bd2051da895a31, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.011528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.025545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1226/2455] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:25 2022] PUT /identity/v3/projects/c56b20a8390c487db3747378dcd79259/users/f4c2224dbf1e49a3b3bd2051da895a31/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:26.035590 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.035884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.036165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.063818 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] MFA Rules not processed for user `f4c2224dbf1e49a3b3bd2051da895a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:26.065019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:26.128567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1230/2456] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:26.133523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.194279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b47e6-0bd9-440c-8551-dcddaa334a02 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.198866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.199493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.199779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.200054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.200745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.202768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.227616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1227/2457] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:26 2022] POST /identity/v3/projects => generated 403 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:26.231535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.242366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531699b-c40f-45d5-9df7-42264ea49ab8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.277127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:26.302132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.302809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.303099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.303411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.304112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.306029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.326100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1231/2458] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:26 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:26.330031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.340951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf585e-c0c2-4a3d-b8a2-f69bdacab8f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.345653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.346374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.346671 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.346958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.347745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.349357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.359852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1228/2459] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:26.363844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.373689 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd2397-d2f9-4f0b-8140-2c44a37fc121 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.378517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.379440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.379729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.380008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] PATH_INFO: `/v3/projects/05b0ff2983da4b4abd0442340114b028/users/f3fab07829e04f6b9b10d0319fd6a016/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.395389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b0ff2983da4b4abd0442340114b028, user_id=f3fab07829e04f6b9b10d0319fd6a016, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.404406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.417636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1232/2460] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:26 2022] PUT /identity/v3/projects/05b0ff2983da4b4abd0442340114b028/users/f3fab07829e04f6b9b10d0319fd6a016/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:26.421432 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.431305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e85e1b-c5aa-4da2-bebf-d3ae2b276f43 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.455158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:26.488884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.489583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.489874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.490649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.491414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.493031 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d2c0b-3c9d-4709-bfc7-bda8e98e8150 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.502598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1229/2461] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:26.507172 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.517330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a63063-9aa0-468b-987d-253ae29d356c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.521932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.522812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.523102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.523415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] PATH_INFO: `/v3/projects/05b0ff2983da4b4abd0442340114b028/users/f3fab07829e04f6b9b10d0319fd6a016/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.525631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b0ff2983da4b4abd0442340114b028, user_id=f3fab07829e04f6b9b10d0319fd6a016, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.528281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.538322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1233/2462] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:26 2022] PUT /identity/v3/projects/05b0ff2983da4b4abd0442340114b028/users/f3fab07829e04f6b9b10d0319fd6a016/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:26.548098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.548394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.548708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.575549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] MFA Rules not processed for user `f3fab07829e04f6b9b10d0319fd6a016`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:26.576755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:26.642824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1230/2463] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:26.653527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.655209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:26.711701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6ff59-e69e-45ae-8607-124829dc1500 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.715256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.715883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.716165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.716441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.725222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.726635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45100c8f-3eed-40e0-bd2f-14cfe1064e24 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.735831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1234/2464] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:26.863545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:26.924274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92fec-b454-4864-8988-1155f74d48f2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:26.929324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:26.930190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:26.930556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:26.930841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] PATH_INFO: `/v3/users/f4c2224dbf1e49a3b3bd2051da895a31` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:26.932067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c2224dbf1e49a3b3bd2051da895a31)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:26.933992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:26.978133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c2224dbf1e49a3b3bd2051da895a31'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:26.978716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c2224dbf1e49a3b3bd2051da895a31'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:26.981179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:26.986311 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c2224dbf1e49a3b3bd2051da895a31'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:26.999870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c2224dbf1e49a3b3bd2051da895a31'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.006511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c2224dbf1e49a3b3bd2051da895a31'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.012350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1231/2465] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:26 2022] DELETE /identity/v3/users/f4c2224dbf1e49a3b3bd2051da895a31 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:27.016914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:27.017393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:27.017491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:27.034874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:27.036107 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:27.094908 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1235/2466] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:27.105226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:27.106629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:27.162827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfba24b-883b-4be2-a2ca-2fd86af83b9b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:27.166511 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:27.167157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:27.169298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:27.169594 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:27.179302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:27.180644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:27.190475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1232/2467] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:27.454303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:27.455749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:27.470318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433b84b-735e-4a53-806d-7df8d6b2745e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:27.475746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:27.476751 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:27.477041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:27.477313 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] PATH_INFO: `/v3/projects/c56b20a8390c487db3747378dcd79259` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:27.479117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56b20a8390c487db3747378dcd79259)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:27.480658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:27.516295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:27.534766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56b20a8390c487db3747378dcd79259', 'request_id': 'req-f08a68f7-6626-4151-bc5e-b2bda1d281d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.541591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56b20a8390c487db3747378dcd79259', 'request_id': 'req-f08a68f7-6626-4151-bc5e-b2bda1d281d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.547396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56b20a8390c487db3747378dcd79259', 'request_id': 'req-f08a68f7-6626-4151-bc5e-b2bda1d281d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.560358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] The token cache is being invalidate because project c56b20a8390c487db3747378dcd79259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:27.560653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.561277 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.562829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1236/2468] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:27 2022] DELETE /identity/v3/projects/c56b20a8390c487db3747378dcd79259 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:27.566837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:27.576584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a153272-1808-4d7e-b60d-bc4123a3d6bd service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:27.636544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:27.637433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:27.637800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:27.638090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] PATH_INFO: `/v3/users/f3fab07829e04f6b9b10d0319fd6a016` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:27.639318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fab07829e04f6b9b10d0319fd6a016)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:27.641254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:27.690379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fab07829e04f6b9b10d0319fd6a016'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.691189 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fab07829e04f6b9b10d0319fd6a016'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.699864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fab07829e04f6b9b10d0319fd6a016'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.713219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fab07829e04f6b9b10d0319fd6a016'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.722391 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:27.739764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb15f10-e36e-488d-ace6-00411dff923d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fab07829e04f6b9b10d0319fd6a016'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:27.749152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1233/2469] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:27 2022] DELETE /identity/v3/users/f3fab07829e04f6b9b10d0319fd6a016 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:28.020868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.022432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:28.039949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a68f7-6626-4151-bc5e-b2bda1d281d0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.103294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.104193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.104475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.104749 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] PATH_INFO: `/v3/projects/05b0ff2983da4b4abd0442340114b028` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.105985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b0ff2983da4b4abd0442340114b028)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.107949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.154238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b0ff2983da4b4abd0442340114b028', 'request_id': 'req-c481cec1-9ca1-4d39-8929-b1e0b237560a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:28.160221 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b0ff2983da4b4abd0442340114b028', 'request_id': 'req-c481cec1-9ca1-4d39-8929-b1e0b237560a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:28.165631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b0ff2983da4b4abd0442340114b028', 'request_id': 'req-c481cec1-9ca1-4d39-8929-b1e0b237560a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:28.177324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] The token cache is being invalidate because project 05b0ff2983da4b4abd0442340114b028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:28.178137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:28.178747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c481cec1-9ca1-4d39-8929-b1e0b237560a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:28.180292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1237/2470] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:28 2022] DELETE /identity/v3/projects/05b0ff2983da4b4abd0442340114b028 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:28.185332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.185807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.185995 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.204472 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:28.205682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:28.264860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1234/2471] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:28.273575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.274170 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.274455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.293390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:28.294591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:28.302466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1238/2472] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:28.307286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.316789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b560d-242b-4dd3-95b7-294818e25db9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.322005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.322663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.322952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.323257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.323993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:28.324305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.325917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.338220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1235/2473] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:28.342450 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.351866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c98ef-b11e-4276-89bc-11927c2e1f5f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.356447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.357102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.357393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.357707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.358426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.360363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.386060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1239/2474] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:28.390193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.400245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa1cb8-5874-4af6-a1e5-6fc10bfdcb0d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.459377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.460057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.460352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.460632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.461341 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.463312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.482222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1236/2475] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:28.486406 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.497336 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80961765-d197-491d-bc97-52cf6cace0f1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.501981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.502647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.502938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.503245 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.503988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.505578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.515929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1240/2476] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:28.519647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.529664 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31818df7-d1cc-413b-aa6a-d78f7a5ffc3d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.534209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.534863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.535150 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.535458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.536184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.538252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703feae-882b-4aca-ac41-85658aff2088 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.547993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1237/2477] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:28.551831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.558877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:28.562782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e2c7-e274-4571-8bec-a2a6f99df25e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.568363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.569411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.569891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.570342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] PATH_INFO: `/v3/projects/8f9405f2b0a5449193f3d4fb431613ae/users/d2450868b2c44f37aced80ed58cb7472/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.584719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9405f2b0a5449193f3d4fb431613ae, user_id=d2450868b2c44f37aced80ed58cb7472, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.587372 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.600155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1241/2478] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:28 2022] PUT /identity/v3/projects/8f9405f2b0a5449193f3d4fb431613ae/users/d2450868b2c44f37aced80ed58cb7472/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:28.610280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.610576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.610859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.637778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] MFA Rules not processed for user `d2450868b2c44f37aced80ed58cb7472`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:28.638979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:28.682316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:28.703089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1238/2479] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:28.707557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.768291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcada50-9bac-4591-a422-e4e849f9d510 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.773307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.774220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.774504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.774782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.775525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.777447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.803996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1242/2480] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:28.808062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.818818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811127a8-fe05-4e33-8b3d-2bb29de8c55b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.879334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.880024 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.880310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.880587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.881353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.883230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.907160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1239/2481] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:28 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:28.908557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.919628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4fc13-c52d-4e49-acf8-6ee74fcc5b2e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.924132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.924856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.925144 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.926310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.927111 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.928738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.940460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1243/2482] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:28.944681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:28.955191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e268885-094f-4f98-a30c-e70bfbc65ddc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:28.961279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:28.962249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:28.962567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:28.962893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] PATH_INFO: `/v3/projects/75c9298b6f2945c4af82d19c0c48c5da/users/0953ca0a4c754c5faa878d48f97ca701/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:28.979179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c9298b6f2945c4af82d19c0c48c5da, user_id=0953ca0a4c754c5faa878d48f97ca701, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:28.982131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:28.995663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1240/2483] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:28 2022] PUT /identity/v3/projects/75c9298b6f2945c4af82d19c0c48c5da/users/0953ca0a4c754c5faa878d48f97ca701/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:28.999882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:29.012504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e115906-4b98-49b7-b884-3bd57e23c824 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:29.072507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:29.073244 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:29.073479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:29.073792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:29.074601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:29.076330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660a756-a8a6-4075-ae30-7c3752b8f7a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:29.087817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1244/2484] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:29.092138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:29.103942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692459a6-6fd2-4477-8a6a-070d2811d380 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:29.109716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:29.110666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:29.110936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:29.111364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] PATH_INFO: `/v3/projects/75c9298b6f2945c4af82d19c0c48c5da/users/0953ca0a4c754c5faa878d48f97ca701/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:29.114159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c9298b6f2945c4af82d19c0c48c5da, user_id=0953ca0a4c754c5faa878d48f97ca701, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:29.117108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:29.128208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1241/2485] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:29 2022] PUT /identity/v3/projects/75c9298b6f2945c4af82d19c0c48c5da/users/0953ca0a4c754c5faa878d48f97ca701/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:29.139524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:29.139875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:29.140161 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:29.177072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] MFA Rules not processed for user `0953ca0a4c754c5faa878d48f97ca701`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:29.178355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:29.220909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:29.253931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1245/2486] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:29.265239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:29.267202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:29.323943 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b28fd-1039-42b7-bb8e-cbd6f676037d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:29.327474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:29.328115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:29.328401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:29.328678 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:29.338123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:29.339586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:29.348932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1242/2487] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:29.418075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:29.427972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cfe3-de71-45a3-8113-9d6f00dde268 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:29.431983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:29.432647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:29.432933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:29.433211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:29.491404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:29.493195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fdace-3947-43e7-b1e5-d456728fdbc3 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:29.503121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1246/2488] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:33.510519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:33.552249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:33.572509 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107395-f711-46e1-bd86-8c1e04294dc3 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:33.576978 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:33.577842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:33.578122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:33.578395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] PATH_INFO: `/v3/users/d2450868b2c44f37aced80ed58cb7472` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:33.579675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2450868b2c44f37aced80ed58cb7472)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:33.581588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:33.625822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2450868b2c44f37aced80ed58cb7472'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:33.626470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2450868b2c44f37aced80ed58cb7472'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:33.634050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2450868b2c44f37aced80ed58cb7472'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:33.645950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2450868b2c44f37aced80ed58cb7472'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:33.653620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2450868b2c44f37aced80ed58cb7472'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:33.659613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1243/2489] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:33 2022] DELETE /identity/v3/users/d2450868b2c44f37aced80ed58cb7472 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:33.664268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:33.664760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:33.664855 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:33.682090 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:33.683324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:33.740438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1247/2490] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:33.749938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:33.751366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:33.811947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8821b-d70d-402a-a629-cd6f95b6052a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:33.815485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:33.816129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:33.816414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:33.816692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:33.826948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:33.829723 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:33.842287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1244/2491] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:34.113534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:34.115133 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:34.128202 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34dde-1c09-4509-99fd-01950638566d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:34.133360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:34.134225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.135042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.135523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] PATH_INFO: `/v3/projects/8f9405f2b0a5449193f3d4fb431613ae` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.136727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9405f2b0a5449193f3d4fb431613ae)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:34.138623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:34.185459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9405f2b0a5449193f3d4fb431613ae', 'request_id': 'req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.191420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9405f2b0a5449193f3d4fb431613ae', 'request_id': 'req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.197506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9405f2b0a5449193f3d4fb431613ae', 'request_id': 'req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.205486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:34.211147 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] The token cache is being invalidate because project 8f9405f2b0a5449193f3d4fb431613ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:34.211681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.212803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.214741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1248/2492] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:34 2022] DELETE /identity/v3/projects/8f9405f2b0a5449193f3d4fb431613ae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:34.218777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:34.228993 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cc668-c473-4b48-a308-861f60d082eb service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:34.285445 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:34.286329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.286614 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.286889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] PATH_INFO: `/v3/users/0953ca0a4c754c5faa878d48f97ca701` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.288643 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0953ca0a4c754c5faa878d48f97ca701)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:34.290566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:34.292852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:34.336612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0953ca0a4c754c5faa878d48f97ca701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.337190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0953ca0a4c754c5faa878d48f97ca701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.344098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0953ca0a4c754c5faa878d48f97ca701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.356503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0953ca0a4c754c5faa878d48f97ca701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.363304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-20177036-fc5c-4d1c-9c8b-01de861cadaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0953ca0a4c754c5faa878d48f97ca701'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.369542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1245/2493] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:34 2022] DELETE /identity/v3/users/0953ca0a4c754c5faa878d48f97ca701 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:34.643116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:34.644669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:34.660580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e469f-7ae2-44f3-94f7-0899ab58dc55 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:34.718044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:34.718906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.719185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.719483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] PATH_INFO: `/v3/projects/75c9298b6f2945c4af82d19c0c48c5da` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.720641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c9298b6f2945c4af82d19c0c48c5da)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:34.722542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:34.768577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c9298b6f2945c4af82d19c0c48c5da', 'request_id': 'req-a0cc4dc5-ed29-4276-a19b-006b24d555ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.774759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c9298b6f2945c4af82d19c0c48c5da', 'request_id': 'req-a0cc4dc5-ed29-4276-a19b-006b24d555ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.780216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c9298b6f2945c4af82d19c0c48c5da', 'request_id': 'req-a0cc4dc5-ed29-4276-a19b-006b24d555ac'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.792195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] The token cache is being invalidate because project 75c9298b6f2945c4af82d19c0c48c5da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:34.792489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.793108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a0cc4dc5-ed29-4276-a19b-006b24d555ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:34.794625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1249/2494] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:34 2022] DELETE /identity/v3/projects/75c9298b6f2945c4af82d19c0c48c5da => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:34.799522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.799992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.800102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.820112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:34.821312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:34.878697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1246/2495] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:34.886896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.887191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.887517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.905437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:34.906639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:34.914792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1250/2496] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:34.918751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:34.930364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051042-95ea-4466-b69c-a64136246ed4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:34.934946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:34.935649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.936045 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.936329 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.937068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:34.937379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:34.939237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:34.953863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1247/2497] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:34.958100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:34.967923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15b3f4-9156-4c6c-bd39-8caf51ae16b3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:34.972597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:34.973251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:34.973535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:34.973808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:34.974505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:34.976467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.001173 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1251/2498] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:34 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:35.005159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.015642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eeffa-31c9-4e61-b4fc-8d1d17c368e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.070124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.070800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.071100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.071424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.072134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.074109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.089176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:35.097624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1248/2499] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:35 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:35.101261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.111727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90513715-c2f3-4954-b114-2b63fde5e162 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.116786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.117470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.117736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.118010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.118736 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.120365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.130718 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1252/2500] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:35.135199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.145135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b75f5-3869-4ebd-ba03-4c60a21b2789 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.151668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.152348 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.152639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.152923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.153661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.155305 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3552d2-1fca-435f-97de-53e2b240678f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.165941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1249/2501] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:31:35.170157 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.180952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9227b-6273-4e50-8559-68195bde5a03 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.185982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.187042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.187362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.187645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] PATH_INFO: `/v3/projects/c374a2c491eb451fadf6546210b23ab2/users/bc30459ff6b948298d18229b3874cdb1/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.202956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c374a2c491eb451fadf6546210b23ab2, user_id=bc30459ff6b948298d18229b3874cdb1, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.205679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.219850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1253/2502] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:35 2022] PUT /identity/v3/projects/c374a2c491eb451fadf6546210b23ab2/users/bc30459ff6b948298d18229b3874cdb1/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:35.231335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.231638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.231926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.274519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] MFA Rules not processed for user `bc30459ff6b948298d18229b3874cdb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:35.275845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:35.348880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1250/2503] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:35.353905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.382871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:35.422063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37691cca-0224-464b-b99d-35b8c5bf6356 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.426948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.427656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.428571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.428863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.429558 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.431490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.458863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1254/2504] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:35 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:35.461911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.472799 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f88b2a-70df-4acb-94e9-0520e6b62a3d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.530278 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.530970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.531254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.531680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.532226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.534148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.555858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1251/2505] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:35 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:35.560137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.571248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a87b8-cfd8-4987-80f0-c74b760c2f40 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.578492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.579294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.579656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.580009 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.580753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.582352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.593764 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1255/2506] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:35.597948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.608428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d595ab3-a3cb-45ae-9b1a-4ac180df8e35 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.613048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.614005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.614357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.614644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] PATH_INFO: `/v3/projects/e81d0416fc8e409ea4e17b33562574ac/users/b4fa44301250475da47b668202d6ceac/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.629981 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81d0416fc8e409ea4e17b33562574ac, user_id=b4fa44301250475da47b668202d6ceac, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.632728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.647380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1252/2507] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:35 2022] PUT /identity/v3/projects/e81d0416fc8e409ea4e17b33562574ac/users/b4fa44301250475da47b668202d6ceac/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:35.651609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.661826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb66c4f-d768-4c37-9340-b952def1cfe2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.723697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.724382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.725318 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.725613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.726352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.727975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc56e94-031f-460c-9c9f-78c941341d0d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.737802 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1256/2508] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:35.742171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.752750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71800-6db2-4022-af08-518b15c6dc43 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.757578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.758558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.758931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.759244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] PATH_INFO: `/v3/projects/e81d0416fc8e409ea4e17b33562574ac/users/b4fa44301250475da47b668202d6ceac/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.761231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81d0416fc8e409ea4e17b33562574ac, user_id=b4fa44301250475da47b668202d6ceac, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.763895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.775960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1253/2509] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:35 2022] PUT /identity/v3/projects/e81d0416fc8e409ea4e17b33562574ac/users/b4fa44301250475da47b668202d6ceac/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:35.788003 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.788301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.788584 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.824809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] MFA Rules not processed for user `b4fa44301250475da47b668202d6ceac`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:35.826091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:35.902314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1257/2510] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:35.913002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:35.914716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:35.946335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:35.973188 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb025f-cbda-4e1c-88e2-620250cc619a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:35.976850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:35.977495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:35.977784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:35.978065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:35.987042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:35.988513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:35.997916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1254/2511] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:36.092887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:36.102842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678de04b-da32-4b42-b502-0fa80f7554e9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:36.106375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:36.107013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:36.107331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:36.107625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:36.124220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:36.164915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:36.166314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d146af-8028-4d43-8539-fe43c84db824 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:36.175344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1258/2512] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:38.726061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:38.792092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a69a5-04e6-41fa-89fc-d690651ebf1b service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:38.796631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:38.797510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:38.797793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:38.798075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] PATH_INFO: `/v3/users/bc30459ff6b948298d18229b3874cdb1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:38.799283 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc30459ff6b948298d18229b3874cdb1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:38.801191 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:38.852919 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc30459ff6b948298d18229b3874cdb1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:38.854352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc30459ff6b948298d18229b3874cdb1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:38.863639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc30459ff6b948298d18229b3874cdb1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:38.882322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc30459ff6b948298d18229b3874cdb1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:38.890768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc30459ff6b948298d18229b3874cdb1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:38.902079 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1255/2513] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:38 2022] DELETE /identity/v3/users/bc30459ff6b948298d18229b3874cdb1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:38.904314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:38.904875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:38.905122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:38.923629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:38.924882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:38.993696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1259/2514] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:39.001353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:39.002787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:39.076217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae504e47-d863-4a59-97ed-27b6f95940b5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:39.080811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:39.081790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:39.082217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:39.082645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:39.091047 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:39.095652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:39.098649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:39.115423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1256/2515] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:39.481205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:39.483386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:39.496308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc15e9-57b6-474d-8500-74151fa220b9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:39.508183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:39.509082 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:39.509370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:39.509651 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] PATH_INFO: `/v3/projects/c374a2c491eb451fadf6546210b23ab2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:39.510881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c374a2c491eb451fadf6546210b23ab2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:39.514396 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:39.590587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c374a2c491eb451fadf6546210b23ab2', 'request_id': 'req-f4b27a1f-780b-4b9e-8a6e-684d05052137'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.597518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c374a2c491eb451fadf6546210b23ab2', 'request_id': 'req-f4b27a1f-780b-4b9e-8a6e-684d05052137'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.609350 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c374a2c491eb451fadf6546210b23ab2', 'request_id': 'req-f4b27a1f-780b-4b9e-8a6e-684d05052137'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.634066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] The token cache is being invalidate because project c374a2c491eb451fadf6546210b23ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:39.635325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.636310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.641370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1260/2516] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:39 2022] DELETE /identity/v3/projects/c374a2c491eb451fadf6546210b23ab2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:39.642607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:39.671685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60829f-de91-4bac-a140-85e4fc601f34 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:39.760851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:39.761818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:39.762569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:39.762769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] PATH_INFO: `/v3/users/b4fa44301250475da47b668202d6ceac` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:39.764469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fa44301250475da47b668202d6ceac)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:39.766916 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:39.825899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fa44301250475da47b668202d6ceac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.826575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fa44301250475da47b668202d6ceac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.836459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fa44301250475da47b668202d6ceac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.849342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fa44301250475da47b668202d6ceac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.856859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1589688f-22dd-409d-8fea-8ffee49270ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fa44301250475da47b668202d6ceac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:39.863192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1257/2517] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:39 2022] DELETE /identity/v3/users/b4fa44301250475da47b668202d6ceac => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:40.149595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.151115 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:40.169065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b27a1f-780b-4b9e-8a6e-684d05052137 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.211592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:40.229940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.230820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.231100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.231405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] PATH_INFO: `/v3/projects/e81d0416fc8e409ea4e17b33562574ac` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.232588 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81d0416fc8e409ea4e17b33562574ac)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.234526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.286645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81d0416fc8e409ea4e17b33562574ac', 'request_id': 'req-c807139a-9497-4493-b510-2dfe3e77e16b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:40.293530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81d0416fc8e409ea4e17b33562574ac', 'request_id': 'req-c807139a-9497-4493-b510-2dfe3e77e16b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:40.299688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81d0416fc8e409ea4e17b33562574ac', 'request_id': 'req-c807139a-9497-4493-b510-2dfe3e77e16b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:40.318075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] The token cache is being invalidate because project e81d0416fc8e409ea4e17b33562574ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:40.318402 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:40.319031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c807139a-9497-4493-b510-2dfe3e77e16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:40.320673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1261/2518] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:40 2022] DELETE /identity/v3/projects/e81d0416fc8e409ea4e17b33562574ac => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:40.325671 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.325966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.326253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.345523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:40.347598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:40.412109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1258/2519] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:40.419797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.420169 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.420455 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.440607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:40.441826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:40.450471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1262/2520] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:31:40.454858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.465181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504500ab-0969-48bc-bff0-9b9a34399802 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.469809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.470481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.470774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.471054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.471835 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:40.472234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.473837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.483934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:40.491698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1259/2521] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:40.492757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.502490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b724d2-5635-48e5-b5fa-c85970f482ac None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.507946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.508615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.508948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.509231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.509937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.511876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.541945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1263/2522] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:40.543108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.569756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be16c1a-9ca9-4395-bcf6-52625eaa54c7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.664945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.665718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.666073 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.666448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.667380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.669317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.695724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1260/2523] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:40.699365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.713314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b475-2da9-49b9-be3a-ae3ac3d6853b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.719782 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.721171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.722014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.722014 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.722772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.724527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.741325 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1264/2524] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:40.741485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.751153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5412f-509e-4d31-b1e9-264f372c8f10 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.756380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.757193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.757479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.757751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.758482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.760104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3549-3086-4ee9-ad79-0dfcb2f37e35 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.773954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1261/2525] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:40.774979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:40.785646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2e90c-3d77-4dc4-87df-55837ff33ab8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:40.790645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:40.791577 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.791864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.792128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] PATH_INFO: `/v3/projects/714e6b5c3ce14eeb8c3647edf085623e/users/c3d265a62f09428fa4aa4c2ea7c59cec/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.808903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=714e6b5c3ce14eeb8c3647edf085623e, user_id=c3d265a62f09428fa4aa4c2ea7c59cec, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:40.817140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:40.834317 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1265/2526] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:40 2022] PUT /identity/v3/projects/714e6b5c3ce14eeb8c3647edf085623e/users/c3d265a62f09428fa4aa4c2ea7c59cec/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:40.844380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:40.844680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:40.844965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:40.872927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] MFA Rules not processed for user `c3d265a62f09428fa4aa4c2ea7c59cec`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:40.874161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:40.938507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1262/2527] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:40.942886 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.006815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f575382-d3e0-47cf-a58c-58d8a24bc20b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.011552 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.012222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.012511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.012794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.013506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.015443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.028822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:41.046824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1266/2528] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:40 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:41.047931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.059136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d82a9-0f40-4087-b44f-571019a0841a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.118653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.119440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.119732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.120009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.120708 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.122625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.138771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:41.145808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1263/2529] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:41 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:41.149592 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.160674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1536387-355b-4155-94cb-13eab8af9256 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.166303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.166985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.167312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.167597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.168328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.169948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.181504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1267/2530] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:41.185576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.198829 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1137e-67e6-4dc2-aa41-983be77d9d4b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.203771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.204687 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.204971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.205248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] PATH_INFO: `/v3/projects/bedd6e056ad64627a25d719268d44dc6/users/dc5938bb2b534f34b9317352645004b9/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.220644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedd6e056ad64627a25d719268d44dc6, user_id=dc5938bb2b534f34b9317352645004b9, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.223326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.236886 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1264/2531] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:41 2022] PUT /identity/v3/projects/bedd6e056ad64627a25d719268d44dc6/users/dc5938bb2b534f34b9317352645004b9/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:41.241390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.256982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bfc2c-408e-4c18-81c1-e26a1ad47540 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.321537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.322510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.322673 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.322938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.324252 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.326165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec90be-4891-40e3-bc8b-6afc7a276a49 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.336630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1268/2532] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:41.340750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.353037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895dfb1-45ff-498a-ba9b-a22bc97f88b4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.357736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.358631 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.358956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.359268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] PATH_INFO: `/v3/projects/bedd6e056ad64627a25d719268d44dc6/users/dc5938bb2b534f34b9317352645004b9/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.361419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedd6e056ad64627a25d719268d44dc6, user_id=dc5938bb2b534f34b9317352645004b9, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.364227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.375644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1265/2533] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:41 2022] PUT /identity/v3/projects/bedd6e056ad64627a25d719268d44dc6/users/dc5938bb2b534f34b9317352645004b9/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:41.386948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.387278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.387568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.413869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] MFA Rules not processed for user `dc5938bb2b534f34b9317352645004b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:41.415086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:41.484929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1269/2534] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:41.497372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.499100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:41.559048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c3166-438e-4ef6-bc5d-e6e3a62458a1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.562694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.563380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.563694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.563977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.573225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.574615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.584535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1266/2535] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:41.659310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:41.669862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135105ba-db0a-4dd4-ae8d-9350de6b0c36 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:41.673457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:41.674095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:41.674395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:41.674655 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:41.730741 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:41.732188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:41.740822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1270/2536] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:44.919374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:44.920874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:44.982792 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f358e4-8c01-4d24-a96b-71c171007bbf service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:44.986972 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:44.987729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:44.988035 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:44.988310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:44.990207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:45.056092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:45.057543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b56ca9-04c4-4864-a088-81b0f7ebfb95 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:45.067237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1267/2537] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:46.391445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:46.398859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:46.406073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931e3e8-056c-4472-9d25-145c9e9ed8e7 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:46.413185 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:46.414198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:46.414485 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:46.414754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] PATH_INFO: `/v3/users/c3d265a62f09428fa4aa4c2ea7c59cec` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:46.415993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d265a62f09428fa4aa4c2ea7c59cec)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:46.417894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:46.464804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d265a62f09428fa4aa4c2ea7c59cec'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:46.477074 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d265a62f09428fa4aa4c2ea7c59cec'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:46.485373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d265a62f09428fa4aa4c2ea7c59cec'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:46.493414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d265a62f09428fa4aa4c2ea7c59cec'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:46.494060 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d265a62f09428fa4aa4c2ea7c59cec'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:46.500645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1271/2538] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:46 2022] DELETE /identity/v3/users/c3d265a62f09428fa4aa4c2ea7c59cec => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:46.504514 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:46.505002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:46.505275 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:46.549810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:46.551923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:46.567946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:46.650282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1268/2539] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:46.665759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:46.671167 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:46.753790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b824fc3-e691-4426-b735-9c7ba49d443b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:46.759062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:46.759500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:46.759792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:46.760410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:46.774030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:46.775568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:46.788054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1272/2540] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:47.512364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:47.515182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:47.534264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c5804-90bb-4afd-b5e5-75d71440412e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:47.539130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:47.540113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:47.540467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:47.540816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] PATH_INFO: `/v3/projects/714e6b5c3ce14eeb8c3647edf085623e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:47.549285 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] RBAC: Authorizing `identity:delete_project(project_id=714e6b5c3ce14eeb8c3647edf085623e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:47.551293 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:47.601108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714e6b5c3ce14eeb8c3647edf085623e', 'request_id': 'req-47793e16-3109-4cb9-9d24-b01bd5f9520a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.608556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714e6b5c3ce14eeb8c3647edf085623e', 'request_id': 'req-47793e16-3109-4cb9-9d24-b01bd5f9520a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.615012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714e6b5c3ce14eeb8c3647edf085623e', 'request_id': 'req-47793e16-3109-4cb9-9d24-b01bd5f9520a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.625304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] The token cache is being invalidate because project 714e6b5c3ce14eeb8c3647edf085623e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:47.625596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.626268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.627888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1269/2541] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:47 2022] DELETE /identity/v3/projects/714e6b5c3ce14eeb8c3647edf085623e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:47.632375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:47.641924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41a59a-c460-44c5-9d2f-c757aec4efe4 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:47.701153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:47.702045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:47.702330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:47.702607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] PATH_INFO: `/v3/users/dc5938bb2b534f34b9317352645004b9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:47.703931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5938bb2b534f34b9317352645004b9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:47.705924 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:47.744054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:47.780497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5938bb2b534f34b9317352645004b9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.800695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5938bb2b534f34b9317352645004b9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.815023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5938bb2b534f34b9317352645004b9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.824612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5938bb2b534f34b9317352645004b9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.825694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42833fca-fcc8-4719-a374-f2a71b7c8bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5938bb2b534f34b9317352645004b9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:47.838174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1273/2542] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:47 2022] DELETE /identity/v3/users/dc5938bb2b534f34b9317352645004b9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:48.279636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.281389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:48.310697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47793e16-3109-4cb9-9d24-b01bd5f9520a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.401011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.402365 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.402744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.403040 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] PATH_INFO: `/v3/projects/bedd6e056ad64627a25d719268d44dc6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.404352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedd6e056ad64627a25d719268d44dc6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.407099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.457823 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:48.467097 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedd6e056ad64627a25d719268d44dc6', 'request_id': 'req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:48.479581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedd6e056ad64627a25d719268d44dc6', 'request_id': 'req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:48.488042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedd6e056ad64627a25d719268d44dc6', 'request_id': 'req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:48.500292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] The token cache is being invalidate because project bedd6e056ad64627a25d719268d44dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:48.500663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:48.501427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-790b25cc-3fb7-46fe-9fa0-2921b14b2e8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:48.504361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1270/2543] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:48 2022] DELETE /identity/v3/projects/bedd6e056ad64627a25d719268d44dc6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:48.508528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.508822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.509131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.529952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:48.531267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:48.597431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1274/2544] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:48.605982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.606483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.606599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.626718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:48.628092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:48.636683 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1271/2545] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:48.641022 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.651789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b90f1-d503-4ab4-ae9c-854bd2fd5fc4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.663545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.664471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.664750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.665038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.665796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:48.666122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.668225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.681896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1275/2546] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:48.686390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.696839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f715d-509f-48f0-b910-457ee92ea109 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.702988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.703785 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.704145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.704504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.705599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.707334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.737394 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1272/2547] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:48 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:48.741882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.754218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f840f-a5e4-406f-bdd6-952533e6edd0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.834687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.835897 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.836296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.837242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.841023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.843449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.869680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1276/2548] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:48 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:48.871476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.882287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2d175-d9a5-4931-89b3-cd2387a88938 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.886908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.887669 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.887929 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.888258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.889124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.890816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.903153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1273/2549] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:48.909339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.910653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.912944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:48.922114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b828-7b7e-4f04-8c7d-b600cb2f7bd8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.925631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09008-b92c-4e87-aaba-84fd923078e8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.926915 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.927855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.928224 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.928599 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.929410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.930421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.931127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.931343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.931639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.931922 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] PATH_INFO: `/v3/users/68676523b99247bf8f5b9a7553e67f26` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.933134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68676523b99247bf8f5b9a7553e67f26)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.935067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:48.941417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:48.941822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1274/2550] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:48.946103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:48.957005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae501d93-2578-4466-bca8-82c2ebb53f6b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:48.963312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:48.964156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:48.964526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:48.964720 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] PATH_INFO: `/v3/projects/b1671896c6514a6196f643286c7fcc24/users/0aad5e05ce324cf694f07412791082d4/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:48.988220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1671896c6514a6196f643286c7fcc24, user_id=0aad5e05ce324cf694f07412791082d4, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:48.991017 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda2be1-5b10-4fb1-8495-68d27bec4bea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.000270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68676523b99247bf8f5b9a7553e67f26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.008864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1275/2551] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:48 2022] PUT /identity/v3/projects/b1671896c6514a6196f643286c7fcc24/users/0aad5e05ce324cf694f07412791082d4/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:49.019662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.020178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.020748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.022083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68676523b99247bf8f5b9a7553e67f26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.032549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68676523b99247bf8f5b9a7553e67f26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.042477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68676523b99247bf8f5b9a7553e67f26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.043312 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68676523b99247bf8f5b9a7553e67f26'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.048706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] MFA Rules not processed for user `0aad5e05ce324cf694f07412791082d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:49.050009 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.052468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1277/2552] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:48 2022] DELETE /identity/v3/users/68676523b99247bf8f5b9a7553e67f26 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:49.124400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1276/2553] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:49.134499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.135880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.204575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994ab2d-49c2-47b3-9cff-43f503a430a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.209982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.210649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.210937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.211305 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.212005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.213925 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.237763 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.239718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.243188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1278/2554] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:49.244307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.261946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e67817-1e22-4692-b502-f0ec3223721c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.308006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:49.323533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387e17f-eacf-4389-9d14-3cdd968d9094 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.326942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.327861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.328144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.328419 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] PATH_INFO: `/v3/projects/e9fb04db8ac9464086cd7f3362e97455` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.328530 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.329096 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.329388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.329600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fb04db8ac9464086cd7f3362e97455)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.329703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.330365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.331610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.332308 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.348899 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:49.356063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1279/2555] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:49.359863 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.371852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fea66-d34c-469f-ab34-9692210d370e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.377872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.378617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.378972 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.379365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.380181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.382596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.388937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fb04db8ac9464086cd7f3362e97455', 'request_id': 'req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.392888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1280/2556] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:31:49.397150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.402169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fb04db8ac9464086cd7f3362e97455', 'request_id': 'req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.409897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fb04db8ac9464086cd7f3362e97455', 'request_id': 'req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.410057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786c5d9-4735-4d26-8e1c-c6cfd0449fae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.420344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] The token cache is being invalidate because project e9fb04db8ac9464086cd7f3362e97455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:49.420634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.421298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b513ef2f-f0b0-40f7-b5b1-5dbb94494ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:49.423142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1277/2557] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:49 2022] DELETE /identity/v3/projects/e9fb04db8ac9464086cd7f3362e97455 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:49.428287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cc12-801f-4641-b851-6de219a9537d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.428577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cc12-801f-4641-b851-6de219a9537d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.428859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cc12-801f-4641-b851-6de219a9537d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.447676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0e63cc12-801f-4641-b851-6de219a9537d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:49.448961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0e63cc12-801f-4641-b851-6de219a9537d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.467606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.468497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.468789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.469064 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] PATH_INFO: `/v3/projects/f75bb711464a406f812fdca74a68bbd9/users/10e8cf64e03245d9b32f746dbb463845/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.486397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75bb711464a406f812fdca74a68bbd9, user_id=10e8cf64e03245d9b32f746dbb463845, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.490766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.506212 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1281/2558] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:49 2022] PUT /identity/v3/projects/f75bb711464a406f812fdca74a68bbd9/users/10e8cf64e03245d9b32f746dbb463845/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:49.510046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.512112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1278/2559] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:31:49.512259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.519410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.519717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.520113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.539501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:49.540712 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.591901 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa070314-7aaf-4d29-a8e4-e72a66360ab9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.600134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.600824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.601545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.601545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.602165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.604728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.611676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1279/2560] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:31:49.616389 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.619531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1282/2561] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:31:49.620808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.626302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3765cd-2a06-4ce3-a6f7-a06306e498e3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.631598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860602d1-356a-4a41-99ef-1750a096cf45 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.632214 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.633096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.633506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.633790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.634533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:49.634842 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.636497 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.637042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.638067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.638365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.638649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] PATH_INFO: `/v3/projects/f75bb711464a406f812fdca74a68bbd9/users/10e8cf64e03245d9b32f746dbb463845/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.641457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75bb711464a406f812fdca74a68bbd9, user_id=10e8cf64e03245d9b32f746dbb463845, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.644370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4b64-fa92-4072-a55c-13a2381d16eb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.645741 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:49.650075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1280/2562] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:49.654152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.656952 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1283/2563] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:49 2022] PUT /identity/v3/projects/f75bb711464a406f812fdca74a68bbd9/users/10e8cf64e03245d9b32f746dbb463845/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:49.668065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f0f6a-6e50-4eae-bf51-d254ca22cdbf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.723539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.723979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.724393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.734830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.735554 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.735849 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.736136 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.736858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.738775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.757949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] MFA Rules not processed for user `10e8cf64e03245d9b32f746dbb463845`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:49.759171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.770880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1281/2564] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:49.775444 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.809582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:49.839282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1284/2565] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:31:49.848758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d9bd1-3ba1-4ca3-a638-e82f5227bfb5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.850296 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.852102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.854496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.855267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.855619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.856018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.856788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.858742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.887335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1282/2566] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:49 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:49.888819 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.899984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884be58e-cae0-4901-8b8f-a8f8dc7c44b0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.904810 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.905476 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.905759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.906036 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.906947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.908424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.919591 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1283/2567] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:49.923043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.924742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed24692-8905-4a71-b215-8ce6d641d441 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.936002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568f66-7e67-41db-b09a-68a26a67e9f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.941096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.941875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.942249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.942598 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.943486 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:49.945161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:49.962582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1284/2568] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:31:49.963627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:49.975160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa283bb-86eb-4dc0-ac5c-e16076bafcb9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:49.980056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.980950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.981234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.981518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] PATH_INFO: `/v3/projects/bfdc61da3c92434e9fc67d25f8d79e3a/users/e99637c101c04d7b8d286eb0a4616732/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.988511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:49.989603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:49.989603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:49.989856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:49.992231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:49.998116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdc61da3c92434e9fc67d25f8d79e3a, user_id=e99637c101c04d7b8d286eb0a4616732, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:50.000959 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba61e0-4e41-4482-9582-159160485904 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:50.024042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1285/2569] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:49 2022] PUT /identity/v3/projects/bfdc61da3c92434e9fc67d25f8d79e3a/users/e99637c101c04d7b8d286eb0a4616732/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:50.026392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:50.026756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:50.027104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:50.056337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:50.056585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] MFA Rules not processed for user `e99637c101c04d7b8d286eb0a4616732`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:50.057638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:50.058142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:50.070006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1285/2570] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:31:50.074882 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:50.130534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1286/2571] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:50.142927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:50.144496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:50.156829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace0a7a-0b3f-4bde-9c8e-57eee11496d2 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:50.211635 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:50.212299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:50.212589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:50.212873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:50.218282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:50.223987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:50.226586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f8f14-463d-4953-b324-58fd23d594fd service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:50.251750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1286/2572] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:52.662060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:52.725423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53f082-2ca5-4899-959f-5e3dfd165574 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:52.729883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:52.730755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:52.731043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:52.731353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] PATH_INFO: `/v3/users/0aad5e05ce324cf694f07412791082d4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:52.732540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aad5e05ce324cf694f07412791082d4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:52.734467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:52.781261 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aad5e05ce324cf694f07412791082d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:52.781862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aad5e05ce324cf694f07412791082d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:52.789067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aad5e05ce324cf694f07412791082d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:52.800887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aad5e05ce324cf694f07412791082d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:52.808106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aad5e05ce324cf694f07412791082d4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:52.813874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1287/2573] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:52 2022] DELETE /identity/v3/users/0aad5e05ce324cf694f07412791082d4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:52.818388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:52.818845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:52.818958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:52.836625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:52.837838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:52.895381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1287/2574] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:52.905926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:52.907395 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:52.949171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:52.966118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe03e53-1178-44c5-9e89-d61ed711a4b2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:52.969818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:52.970505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:52.970793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:52.971076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:52.981735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:52.983174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:52.992992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1288/2575] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:31:53.269715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:53.272508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:53.286015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7d749-2388-43d1-9fe7-da8938c61606 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:53.291243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:53.292248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:53.292604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:53.301896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] PATH_INFO: `/v3/projects/b1671896c6514a6196f643286c7fcc24` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:53.301896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1671896c6514a6196f643286c7fcc24)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:53.301896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:53.367537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1671896c6514a6196f643286c7fcc24', 'request_id': 'req-09efbc04-14ac-440f-be6a-05f3c24c4e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.375025 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1671896c6514a6196f643286c7fcc24', 'request_id': 'req-09efbc04-14ac-440f-be6a-05f3c24c4e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.382603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1671896c6514a6196f643286c7fcc24', 'request_id': 'req-09efbc04-14ac-440f-be6a-05f3c24c4e14'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.397562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] The token cache is being invalidate because project b1671896c6514a6196f643286c7fcc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:53.397860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.398885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.400517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1288/2576] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:53 2022] DELETE /identity/v3/projects/b1671896c6514a6196f643286c7fcc24 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:53.404597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:53.414401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449048f-4956-4ac6-b020-d2392c734f94 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:53.470310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:53.471185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:53.471575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:53.471858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] PATH_INFO: `/v3/users/10e8cf64e03245d9b32f746dbb463845` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:53.473046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e8cf64e03245d9b32f746dbb463845)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:53.475001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:53.518172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e8cf64e03245d9b32f746dbb463845'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.518860 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e8cf64e03245d9b32f746dbb463845'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.526767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e8cf64e03245d9b32f746dbb463845'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.538424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e8cf64e03245d9b32f746dbb463845'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.545072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d3411768-069d-4f7a-a865-bff30f805505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e8cf64e03245d9b32f746dbb463845'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.550880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1289/2577] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:53 2022] DELETE /identity/v3/users/10e8cf64e03245d9b32f746dbb463845 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:53.826496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:53.827984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:53.850253 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efbc04-14ac-440f-be6a-05f3c24c4e14 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:53.909861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:53.910734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:53.911024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:53.911339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] PATH_INFO: `/v3/projects/f75bb711464a406f812fdca74a68bbd9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:53.913045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75bb711464a406f812fdca74a68bbd9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:53.914958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:53.936131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:53.962862 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75bb711464a406f812fdca74a68bbd9', 'request_id': 'req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.970135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75bb711464a406f812fdca74a68bbd9', 'request_id': 'req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.976345 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75bb711464a406f812fdca74a68bbd9', 'request_id': 'req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.988483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] The token cache is being invalidate because project f75bb711464a406f812fdca74a68bbd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:53.988774 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.989403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-86c76e7c-20c7-405c-8dbe-7a6830fe0cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:53.991477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1289/2578] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:53 2022] DELETE /identity/v3/projects/f75bb711464a406f812fdca74a68bbd9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:53.996298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:53.996755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:53.996883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.014464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:54.015701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:54.066294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:54.081947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1290/2579] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:54.083789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.084081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.084363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.103508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:54.104753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:54.112715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1290/2580] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:54.117507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.127168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4e1c-c03d-42f2-9a80-fae04a60d143 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.131717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.132373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.132662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.132941 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.133672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:54.134011 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.135646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.146950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1291/2581] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:54.151601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.160815 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb020c-37bd-4eb5-84db-59abf8c10726 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.165934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.166585 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.166874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.167154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.167883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.169768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.195380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1291/2582] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:54 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:54.199055 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.208960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270769bb-4a88-4681-98eb-a616e3359277 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.264667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.265352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.265642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.265930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.266639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.268603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.289212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1292/2583] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:54 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:54.293266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.303733 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d193-6139-482a-b644-3852ea5b02fc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.308247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.308904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.309194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.309471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.310199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.311817 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.322017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1292/2584] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:31:54.325724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.336054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f37379-12ec-41e8-a4a0-fe4af00510c4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.340540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.341202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.341491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.341770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.342499 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.344118 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448f4a4-df50-474f-91d7-3d85a1b20098 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.354440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1293/2585] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:31:54.358295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.368448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51476cd3-8407-4f97-b745-09a21929e0f5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.373607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.374491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.374775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.375051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] PATH_INFO: `/v3/projects/68634367b5ae4affb322a6b333bb1d59/users/5bd62ba0044241f3aa6d6c8c8a3f389f/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.389566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=68634367b5ae4affb322a6b333bb1d59, user_id=5bd62ba0044241f3aa6d6c8c8a3f389f, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.392231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.407036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1293/2586] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:31:54 2022] PUT /identity/v3/projects/68634367b5ae4affb322a6b333bb1d59/users/5bd62ba0044241f3aa6d6c8c8a3f389f/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:54.419008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1691be-14ec-4bef-930a-1cdfceb7ca22 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.419561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1691be-14ec-4bef-930a-1cdfceb7ca22 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.419850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1691be-14ec-4bef-930a-1cdfceb7ca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.446269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5d1691be-14ec-4bef-930a-1cdfceb7ca22 None None] MFA Rules not processed for user `5bd62ba0044241f3aa6d6c8c8a3f389f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:54.447548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5d1691be-14ec-4bef-930a-1cdfceb7ca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:54.512937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1294/2587] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:31:54.524712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:54.526426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:54.584311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a37ce-094b-4226-90f8-6d13594a08cc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:54.588143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:54.588756 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:54.589042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:54.589322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:54.597483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:54.598888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:54.608445 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1294/2588] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:56.896884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:56.897418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:56.897536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:56.916026 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:56.917237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:56.966705 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:56.975540 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1295/2589] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:57.023977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:57.025411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:57.073382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:57.095363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3e2d8-dfac-4ea8-8261-4e8a987e9023 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:57.100847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:57.101508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:57.101798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:57.102078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:57.110301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:57.111898 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d8b86-0501-4c37-9d31-f9a99d2041b9 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:57.120337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1295/2590] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:31:57.223474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:57.312867 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b54fbf-c96c-4f15-a896-0b3f1d0fab6c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:57.319496 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:57.320644 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:57.321131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:57.321624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] PATH_INFO: `/v3/users/5bd62ba0044241f3aa6d6c8c8a3f389f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:57.322926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd62ba0044241f3aa6d6c8c8a3f389f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:57.325111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:57.404280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd62ba0044241f3aa6d6c8c8a3f389f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:57.415852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd62ba0044241f3aa6d6c8c8a3f389f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:57.428124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd62ba0044241f3aa6d6c8c8a3f389f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:57.472543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd62ba0044241f3aa6d6c8c8a3f389f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:57.481337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd62ba0044241f3aa6d6c8c8a3f389f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:57.488909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1296/2591] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:57 2022] DELETE /identity/v3/users/5bd62ba0044241f3aa6d6c8c8a3f389f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:57.493382 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:57.493775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:57.494124 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:57.517451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:57.517451 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:57.611203 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1296/2592] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:57.622199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:57.623747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:57.698742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab71fa7-1f22-407d-8526-247e4283940f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:57.702513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:57.703454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:57.703769 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:57.704028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:57.714930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:57.716563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:57.736092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1297/2593] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:58.061740 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:58.063330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:58.085039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950339d-e80e-45d5-955b-4ec7284a68cb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:58.094996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:58.096049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:58.096436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:58.096775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] PATH_INFO: `/v3/projects/68634367b5ae4affb322a6b333bb1d59` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:58.098214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68634367b5ae4affb322a6b333bb1d59)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:58.102435 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:58.165392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68634367b5ae4affb322a6b333bb1d59', 'request_id': 'req-2019c0f7-70a5-4dc9-a128-294ade595975'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.173078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68634367b5ae4affb322a6b333bb1d59', 'request_id': 'req-2019c0f7-70a5-4dc9-a128-294ade595975'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.179480 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68634367b5ae4affb322a6b333bb1d59', 'request_id': 'req-2019c0f7-70a5-4dc9-a128-294ade595975'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.197663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] The token cache is being invalidate because project 68634367b5ae4affb322a6b333bb1d59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:58.198055 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.198746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2019c0f7-70a5-4dc9-a128-294ade595975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.200431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1297/2594] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:58 2022] DELETE /identity/v3/projects/68634367b5ae4affb322a6b333bb1d59 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:58.693772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:58.695890 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:58.716317 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f686cc-a59c-4bb6-b1e7-f46d9c339bc7 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:58.752438 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:58.810582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:58.812456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:58.812747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:58.813950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] PATH_INFO: `/v3/users/e99637c101c04d7b8d286eb0a4616732` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:58.815971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99637c101c04d7b8d286eb0a4616732)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:58.818820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:58.897030 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99637c101c04d7b8d286eb0a4616732'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.897866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99637c101c04d7b8d286eb0a4616732'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.909266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99637c101c04d7b8d286eb0a4616732'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.924610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99637c101c04d7b8d286eb0a4616732'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.932982 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99637c101c04d7b8d286eb0a4616732'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:58.940624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1298/2595] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:31:58 2022] DELETE /identity/v3/users/e99637c101c04d7b8d286eb0a4616732 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:31:58.946359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:58.946732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:58.947020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:58.966463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:58.967723 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:59.025343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:59.035824 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1298/2596] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:59.048731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:59.050182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:59.116767 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524732a-ed1a-4746-9c2b-21902cfc8e57 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:59.122780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:59.124235 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.124522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.124806 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.135562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:59.137088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4be07-6bcf-48cf-97de-848c74b07f88 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:59.149284 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1299/2597] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:31:59.464787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:59.466441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:59.490117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e664f-7c0b-41f6-a0f8-250ea134c9b5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:59.509059 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:59.510004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.510291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.510570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] PATH_INFO: `/v3/projects/bfdc61da3c92434e9fc67d25f8d79e3a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.511852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfdc61da3c92434e9fc67d25f8d79e3a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:59.514010 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:59.573519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfdc61da3c92434e9fc67d25f8d79e3a', 'request_id': 'req-b67a247a-83d7-47b6-93ce-562332199584'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:59.583920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfdc61da3c92434e9fc67d25f8d79e3a', 'request_id': 'req-b67a247a-83d7-47b6-93ce-562332199584'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:59.592374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfdc61da3c92434e9fc67d25f8d79e3a', 'request_id': 'req-b67a247a-83d7-47b6-93ce-562332199584'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:59.608015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] The token cache is being invalidate because project bfdc61da3c92434e9fc67d25f8d79e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:31:59.609486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:59.609486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b67a247a-83d7-47b6-93ce-562332199584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:31:59.614685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1299/2598] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:31:59 2022] DELETE /identity/v3/projects/bfdc61da3c92434e9fc67d25f8d79e3a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:31:59.616268 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.616573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.616862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.636506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:59.637755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:59.682716 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:31:59.708654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1300/2599] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:59.716228 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93be06-c929-4719-8f99-95205e5699db None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.716522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93be06-c929-4719-8f99-95205e5699db None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.716804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93be06-c929-4719-8f99-95205e5699db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.736783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4c93be06-c929-4719-8f99-95205e5699db None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:31:59.738547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4c93be06-c929-4719-8f99-95205e5699db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:31:59.748142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1300/2600] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:31:59.752059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:59.766461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed064845-1f99-441b-ac2d-f5322c7d1c62 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:59.772162 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:59.772843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.773421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.775393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.775393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:31:59.775393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:59.776662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:59.794104 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1301/2601] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:31:59.794776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93be06-c929-4719-8f99-95205e5699db None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:59.811960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93be06-c929-4719-8f99-95205e5699db None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:59.817342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:59.818235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.818622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.819559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.819856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:59.821965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:59.851274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1301/2602] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:31:59 2022] POST /identity/v3/projects => generated 357 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:31:59.855491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:31:59.868605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa54447-49b4-4efe-bbdb-1c5ee176aba5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:31:59.957145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:31:59.957834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:31:59.958119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:31:59.958393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:31:59.959100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:31:59.961090 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:31:59.995290 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1302/2603] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:31:59 2022] POST /identity/v3/users => generated 279 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:31:59.995707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.010893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41b097-2831-4bc2-85fe-2b2bd0a9c23d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.016826 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.017597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.017923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.018200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.018974 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.020747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.037969 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1302/2604] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:32:00.039525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.050754 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c38f-83b2-4d8d-8ad6-ab5a0bbafcc8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.056777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.057602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.057893 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.058172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.058914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.060749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdca4c-a435-4b35-b0b3-26b5875bce76 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.075088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1303/2605] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:00.079971 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.095280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda15546-4cc2-4143-92d3-29128e68cc85 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.102524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.103958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.104393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.106104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] PATH_INFO: `/v3/projects/24d72f011ba347a38759bb7056985f8c/users/14dfbd60633646a382847049f6247332/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.132230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d72f011ba347a38759bb7056985f8c, user_id=14dfbd60633646a382847049f6247332, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.134944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.148891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1303/2606] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:00 2022] PUT /identity/v3/projects/24d72f011ba347a38759bb7056985f8c/users/14dfbd60633646a382847049f6247332/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:00.160409 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.160805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.161159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.192135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] MFA Rules not processed for user `14dfbd60633646a382847049f6247332`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:00.193398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:00.283504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1304/2607] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:00.288698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.362557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fe5bb-fcd9-4f44-9e33-4241cfad094a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.371363 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.372140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.372425 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.372707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.373405 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.375946 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.406920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1304/2608] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:00 2022] POST /identity/v3/projects => generated 355 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:00.410831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.424334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eec3c2-511c-4c5f-b512-ba583c3d3f9b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.470887 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:00.495463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.500001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.500385 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.500722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.501814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.503927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.537361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.538366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1305/2609] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:00 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:00.545267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:00.549177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237bdf-7ee5-4182-ac62-223e3f2c2b6d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.556615 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.557932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.558761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.559567 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.560474 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.562270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.579251 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1305/2610] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:00.581677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.592694 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1e132-3d43-4646-b9e8-fb79c8432971 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.598504 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.599549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.599903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.600250 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] PATH_INFO: `/v3/projects/25b7182e42a8472ba723860253b4c056/users/b1d16b6cac504e6faad26cd0611fdc21/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.619046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b7182e42a8472ba723860253b4c056, user_id=b1d16b6cac504e6faad26cd0611fdc21, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.621750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.637352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1306/2611] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:00 2022] PUT /identity/v3/projects/25b7182e42a8472ba723860253b4c056/users/b1d16b6cac504e6faad26cd0611fdc21/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:00.641374 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.653583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b4d2b-67d9-466a-bd87-34efd3701ec5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.716568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.717582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.718553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.719953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.722787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.724510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35918317-391d-4851-985b-b090270c8d76 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.741670 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1306/2612] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:00.743116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.756617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7c4fd-ff4a-4821-adb6-b24fc042eaf6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.761477 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.762446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.762820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.763163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] PATH_INFO: `/v3/projects/25b7182e42a8472ba723860253b4c056/users/b1d16b6cac504e6faad26cd0611fdc21/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.766181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b7182e42a8472ba723860253b4c056, user_id=b1d16b6cac504e6faad26cd0611fdc21, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:00.768917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:00.781653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1307/2613] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:00 2022] PUT /identity/v3/projects/25b7182e42a8472ba723860253b4c056/users/b1d16b6cac504e6faad26cd0611fdc21/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:00.792042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.792461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.792844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:00.822575 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] MFA Rules not processed for user `b1d16b6cac504e6faad26cd0611fdc21`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:00.824080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:00.900994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1307/2614] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:32:00.912663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:00.914414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:00.985325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029719a-0918-4602-a3e1-082b3497966b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:00.989131 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:00.989827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:00.990121 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:00.990528 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:01.000926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:01.002658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818c7c7-4032-49d4-96ad-f3c73b7b4b79 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:01.015532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1308/2615] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:32:01.116800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:01.129381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fea959-bca3-4957-9939-3c1b2ed2a2a8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:01.139532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:01.140366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:01.140666 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:01.141218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:01.216679 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:01.218248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d13b-3306-438d-9d64-ba316cf3b880 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:01.229545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1308/2616] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:01 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:10.064340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.064946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.065336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.067161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1309/2617] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:32:10.078649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494a2d4-4eb4-46ff-83fc-56a815d0250c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.079040 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494a2d4-4eb4-46ff-83fc-56a815d0250c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.079208 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494a2d4-4eb4-46ff-83fc-56a815d0250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.104731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:10.120214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a494a2d4-4eb4-46ff-83fc-56a815d0250c None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:10.121438 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a494a2d4-4eb4-46ff-83fc-56a815d0250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.192534 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1309/2618] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:10.203437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:10.205173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.286992 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e875bf-7191-47ba-86fe-6ab7ff525743 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:10.290853 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f11ea126550c4c4a8892522e01ed149a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:10.291549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.291837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.292117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.302296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:10.303901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427848f-f5c9-410a-84d9-1fcfd891ce02 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:10.314561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1310/2619] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:10.379900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.380303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.380628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.382132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1310/2620] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:32:10.387032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa7e75-e520-4012-aa1c-377df49bfb55 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.387364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa7e75-e520-4012-aa1c-377df49bfb55 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.387656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa7e75-e520-4012-aa1c-377df49bfb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.406369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1aaa7e75-e520-4012-aa1c-377df49bfb55 None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:10.407609 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1aaa7e75-e520-4012-aa1c-377df49bfb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.420052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:10.459802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1311/2621] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:32:10.471299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:10.473002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.486076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaa3e1-9186-46a9-9268-4d657c013755 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:10.489596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:10.490229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.490518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.490798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.497805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:10.499248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a5750-7f5a-4dae-bb4d-63b23de07572 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:10.507642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1311/2622] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:32:10.588957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.589253 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.589541 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.590911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1312/2623] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:32:10.595913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18f54-9686-4047-ba1c-a378a763c266 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.596204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18f54-9686-4047-ba1c-a378a763c266 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.596484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18f54-9686-4047-ba1c-a378a763c266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.614235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5ff18f54-9686-4047-ba1c-a378a763c266 None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:10.615459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ff18f54-9686-4047-ba1c-a378a763c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.624499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1312/2624] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:32:10.635046 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:10.636721 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:10.650793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996528c4-d37c-4f60-8496-0e17e912c5aa None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:10.654652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:10.655337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:10.655630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:10.655913 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:10.666184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:10.667661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:10.678312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1313/2625] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:14.048565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b11b-1378-42fe-8056-a3e595fda94b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:14.049078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b11b-1378-42fe-8056-a3e595fda94b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:14.049370 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b11b-1378-42fe-8056-a3e595fda94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:14.070368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9106b11b-1378-42fe-8056-a3e595fda94b None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:14.071636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9106b11b-1378-42fe-8056-a3e595fda94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:14.132288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1313/2626] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:32:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:14.144211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:14.145620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:14.216718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1222c9-6b39-4ec6-a5b7-fb8627ff772f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:14.221909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:14.222680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:14.222973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:14.223773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:14.231750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:14.233327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94733726-fec2-47c6-bebd-aaa2019031bc service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:14.242089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1314/2627] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:32:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:32:14.892270 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e3fd-31b0-42ca-ad27-ee6b9d480898 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:14.892622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e3fd-31b0-42ca-ad27-ee6b9d480898 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:14.892928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10e3fd-31b0-42ca-ad27-ee6b9d480898 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:14.894821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1314/2628] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:32:14.905005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45417-b8fe-40f1-8f25-6381755f6d0c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:14.905005 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45417-b8fe-40f1-8f25-6381755f6d0c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:14.905301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb45417-b8fe-40f1-8f25-6381755f6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:14.934300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0cb45417-b8fe-40f1-8f25-6381755f6d0c None None] MFA Rules not processed for user `1c6dce92d6724fbf878faba138471bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:14.939058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0cb45417-b8fe-40f1-8f25-6381755f6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:14.974965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1315/2629] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:32:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:14.987044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10e3fd-31b0-42ca-ad27-ee6b9d480898 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:15.000690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10e3fd-31b0-42ca-ad27-ee6b9d480898 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:15.011799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:15.011799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:15.011799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:15.011799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:15.026449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:15.026449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fda60-ce34-41eb-97a8-9d5b136b1a9e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:15.043239 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1315/2630] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:15.191025 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399245d-eb17-41e1-b348-0a992561e025 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:15.192537 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399245d-eb17-41e1-b348-0a992561e025 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:15.193142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399245d-eb17-41e1-b348-0a992561e025 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:15.195174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1316/2631] 10.222.0.46 () {54 vars in 921 bytes} [Wed Jul 6 15:32:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 15:32:15.201980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:15.202571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:15.203030 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:15.232395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] MFA Rules not processed for user `aff09cb4b1ff42c0a011f8e09a3c6c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:15.234772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:15.258036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1316/2632] 10.222.0.46 () {58 vars in 1018 bytes} [Wed Jul 6 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:15.273873 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399245d-eb17-41e1-b348-0a992561e025 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:15.287870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0399245d-eb17-41e1-b348-0a992561e025 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:15.291665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:15.292404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:15.292759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:15.293117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:15.307408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:15.309219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe15cb-be01-4838-ad72-466b178b8450 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:15.325971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1317/2633] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:16.673505 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:16.734789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:16.740189 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417aeeaa-b547-4c70-9653-a0ca1ae9c90d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:16.748216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:16.749254 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:16.749637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:16.749976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] PATH_INFO: `/v3/users/14dfbd60633646a382847049f6247332` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:16.751339 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=14dfbd60633646a382847049f6247332)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:16.753361 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:16.807183 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14dfbd60633646a382847049f6247332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:16.824434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14dfbd60633646a382847049f6247332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:16.833601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14dfbd60633646a382847049f6247332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:16.841362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14dfbd60633646a382847049f6247332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:16.842117 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14dfbd60633646a382847049f6247332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:16.849084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1317/2634] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:16 2022] DELETE /identity/v3/users/14dfbd60633646a382847049f6247332 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:16.853839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:16.854356 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:16.854663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:16.873092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:16.874668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:16.941184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1318/2635] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:16.947776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:16.949263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:17.011580 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7ce5-9feb-4600-b49e-dcf95aa269bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:17.018231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:17.018904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:17.019195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:17.019519 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:17.030470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:17.031960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:17.043955 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1318/2636] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:17.490706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:17.492186 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:17.517991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219fd5-0d8b-43ce-a398-cb3576f22653 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:17.533571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:17.534655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:17.536013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:17.536407 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] PATH_INFO: `/v3/projects/24d72f011ba347a38759bb7056985f8c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:17.538103 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d72f011ba347a38759bb7056985f8c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:17.541234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:17.570077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:17.610336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d72f011ba347a38759bb7056985f8c', 'request_id': 'req-05f63380-bffa-4367-beb8-c822ee568725'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.622441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d72f011ba347a38759bb7056985f8c', 'request_id': 'req-05f63380-bffa-4367-beb8-c822ee568725'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.646700 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d72f011ba347a38759bb7056985f8c', 'request_id': 'req-05f63380-bffa-4367-beb8-c822ee568725'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.662917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] The token cache is being invalidate because project 24d72f011ba347a38759bb7056985f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:17.663308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.663984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.665805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1319/2637] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:17 2022] DELETE /identity/v3/projects/24d72f011ba347a38759bb7056985f8c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:17.669582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:17.679930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2163c-4d68-48db-89fa-d407ce967846 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:17.746054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:17.747002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:17.747319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:17.747604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] PATH_INFO: `/v3/users/b1d16b6cac504e6faad26cd0611fdc21` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:17.748827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d16b6cac504e6faad26cd0611fdc21)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:17.750757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:17.776571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:17.829274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d16b6cac504e6faad26cd0611fdc21'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.851282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d16b6cac504e6faad26cd0611fdc21'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.863773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d16b6cac504e6faad26cd0611fdc21'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.873008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d16b6cac504e6faad26cd0611fdc21'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.873848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1c61f5a1-7e19-41db-b70b-34ad33135629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d16b6cac504e6faad26cd0611fdc21'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:17.881797 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1319/2638] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:17 2022] DELETE /identity/v3/users/b1d16b6cac504e6faad26cd0611fdc21 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:18.229667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.231157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:18.256940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f63380-bffa-4367-beb8-c822ee568725 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.349382 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.350276 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.350570 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.351358 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] PATH_INFO: `/v3/projects/25b7182e42a8472ba723860253b4c056` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.352617 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b7182e42a8472ba723860253b4c056)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.354545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.408942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b7182e42a8472ba723860253b4c056', 'request_id': 'req-68c4e04c-c659-4481-82be-18b8e65d7c0d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:18.417775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b7182e42a8472ba723860253b4c056', 'request_id': 'req-68c4e04c-c659-4481-82be-18b8e65d7c0d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:18.425056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b7182e42a8472ba723860253b4c056', 'request_id': 'req-68c4e04c-c659-4481-82be-18b8e65d7c0d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:18.436814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] The token cache is being invalidate because project 25b7182e42a8472ba723860253b4c056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:18.437106 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:18.437780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-68c4e04c-c659-4481-82be-18b8e65d7c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:18.443553 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1320/2639] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:18 2022] DELETE /identity/v3/projects/25b7182e42a8472ba723860253b4c056 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:18.445471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.445800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.446243 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.469304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:18.470628 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:18.538258 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1320/2640] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:18.546067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.546627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.547127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.569934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:18.571558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:18.581489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1321/2641] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:18.586395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.596491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f653a09-ca04-4164-a06b-3d8af8706402 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.612165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.613121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.614152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.614640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.615605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:32:18.616108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.617933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.651864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1321/2642] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:32:18.654066 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.683060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eaaff-c521-41df-9b9c-8d0a8a28e13d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.692372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.693075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.693368 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.693648 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.694361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.696337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.767661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1322/2643] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:18 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:18.771613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.786583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75d8fa-cdd2-49da-9160-0290464f2b1b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.877717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.878508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.878906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.879811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.880542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.882555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.908135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1322/2644] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:18 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:18.909120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.921798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4e3d-7531-4fea-9df1-e5e7fa63a853 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.927457 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.928163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.928443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.928728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.929482 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.931096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.933601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:18.946749 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1323/2645] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:18.948278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.959035 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916a794-d0c0-4ffe-bfff-c2ce56cc4b65 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.963923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.964601 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:18.964892 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:18.965174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:18.965951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:18.967889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804996f-5bde-4d7b-aa08-a1886413bac0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:18.979257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1323/2646] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:18.983360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:18.994168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb393058-2e7a-4f85-8275-34d37bb5cab8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:18.998917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:18.999878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.000170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.000451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] PATH_INFO: `/v3/projects/638fd8c2253246e985b99763c0c84ce6/users/e5c321d21c0740a1a313d203a76343bd/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.016129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=638fd8c2253246e985b99763c0c84ce6, user_id=e5c321d21c0740a1a313d203a76343bd, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.018815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.041072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1324/2647] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:18 2022] PUT /identity/v3/projects/638fd8c2253246e985b99763c0c84ce6/users/e5c321d21c0740a1a313d203a76343bd/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:19.053389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.053985 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.054459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.062753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:19.092193 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] MFA Rules not processed for user `e5c321d21c0740a1a313d203a76343bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:19.093753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:19.167442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1324/2648] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:32:19.172006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.237706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc993a-1a35-4271-aa9e-336628b116e2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.244048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.245173 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.245660 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.246120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.247727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.249265 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.277212 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1325/2649] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:19 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:19.281278 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.297021 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d9ec9-ba5d-47a7-910b-b07aed3ee1d4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.363496 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.364225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.364517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.364800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.365503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.367465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.395665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1325/2650] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:19 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:19.399852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.411053 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a866ff-29f1-43de-8121-123f7a080c7b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.416007 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.416696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.417134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.417421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.418168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.419866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.436901 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1326/2651] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:19.441154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.451732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e450-22ba-4818-a4c6-dace7efad011 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.456872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.458344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.458888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.458888 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] PATH_INFO: `/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.476814 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6de0e48c1fe48ec8b4623540cff8eca, user_id=2cef7cc5f3c04eeabc4d0d472ffa8fa2, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.479677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.498688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1326/2652] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:19 2022] PUT /identity/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:19.502632 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.514282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d375859-0a63-4413-9c4b-2308e901e22d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.575478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.576163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.576464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.576746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.577493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.579108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acb1c2-ee1f-416e-92bd-83b64075b0a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.597755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1327/2653] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:19.598943 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.612482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399baeb5-a963-473a-9975-4b5e4742d972 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.617786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.618703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.619065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.619291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] PATH_INFO: `/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.621322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6de0e48c1fe48ec8b4623540cff8eca, user_id=2cef7cc5f3c04eeabc4d0d472ffa8fa2, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.623982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.645557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1327/2654] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:19 2022] PUT /identity/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:19.647803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.648156 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.648447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.679271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] MFA Rules not processed for user `2cef7cc5f3c04eeabc4d0d472ffa8fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:19.681023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:19.718213 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:19.775418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1328/2655] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:32:19.783773 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:19.787712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:19.876216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:19.886667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1d93a-5ae7-4116-83ff-e54e21f39ac9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:19.892452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:19.894284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:19.894957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:19.895460 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:19.950544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:19.953667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d1de2-955c-44b3-8435-133742737f04 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:19.991935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1328/2656] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:32:20.078605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:20.196471 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087dbd4-7f6a-4151-b88f-d3977f654f4f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:20.201273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:20.202164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:20.202449 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:20.202775 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] PATH_INFO: `/v3/users/e5c321d21c0740a1a313d203a76343bd` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:20.204099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c321d21c0740a1a313d203a76343bd)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:20.206029 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:20.258603 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c321d21c0740a1a313d203a76343bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.259331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c321d21c0740a1a313d203a76343bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.268794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c321d21c0740a1a313d203a76343bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.286676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c321d21c0740a1a313d203a76343bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.296997 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c321d21c0740a1a313d203a76343bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.303165 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1329/2657] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:20 2022] DELETE /identity/v3/users/e5c321d21c0740a1a313d203a76343bd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:20.307735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653628-2192-45ea-a493-9f68290aa55c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:20.308104 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653628-2192-45ea-a493-9f68290aa55c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:20.308456 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653628-2192-45ea-a493-9f68290aa55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:20.330018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-66653628-2192-45ea-a493-9f68290aa55c None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:20.331283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66653628-2192-45ea-a493-9f68290aa55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:20.398134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1329/2658] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:20.404230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:20.405921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:20.469000 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:20.479197 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14769fe7-de89-40ca-bd1d-fe69dde2ce90 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:20.483956 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:20.484620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:20.484907 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:20.485318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:20.496397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:20.497854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:20.509065 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1330/2659] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:20.831256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66653628-2192-45ea-a493-9f68290aa55c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:20.840568 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-66653628-2192-45ea-a493-9f68290aa55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:20.853076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66653628-2192-45ea-a493-9f68290aa55c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:20.857929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:20.858847 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:20.859067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:20.859791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] PATH_INFO: `/v3/projects/638fd8c2253246e985b99763c0c84ce6` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:20.860945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=638fd8c2253246e985b99763c0c84ce6)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:20.862875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:20.930272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '638fd8c2253246e985b99763c0c84ce6', 'request_id': 'req-8a691595-c700-4d43-ae97-2bf7b4bbaee4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.938563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '638fd8c2253246e985b99763c0c84ce6', 'request_id': 'req-8a691595-c700-4d43-ae97-2bf7b4bbaee4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.946408 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '638fd8c2253246e985b99763c0c84ce6', 'request_id': 'req-8a691595-c700-4d43-ae97-2bf7b4bbaee4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.960211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] The token cache is being invalidate because project 638fd8c2253246e985b99763c0c84ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:20.960502 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.961123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:20.962703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1330/2660] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:20 2022] DELETE /identity/v3/projects/638fd8c2253246e985b99763c0c84ce6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:20.966927 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:20.980114 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d3305-172e-4ab0-a994-0d50a5141d95 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:21.045145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:21.046578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.046864 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.047142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] PATH_INFO: `/v3/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.048568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cef7cc5f3c04eeabc4d0d472ffa8fa2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:21.050946 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:21.116236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cef7cc5f3c04eeabc4d0d472ffa8fa2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.117252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cef7cc5f3c04eeabc4d0d472ffa8fa2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.125220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cef7cc5f3c04eeabc4d0d472ffa8fa2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.142839 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cef7cc5f3c04eeabc4d0d472ffa8fa2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.150770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a33656c9-be7e-4eba-819b-584a83ebf6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cef7cc5f3c04eeabc4d0d472ffa8fa2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.158153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1331/2661] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:20 2022] DELETE /identity/v3/users/2cef7cc5f3c04eeabc4d0d472ffa8fa2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:21.609887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:21.611418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:21.624849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a691595-c700-4d43-ae97-2bf7b4bbaee4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:21.689174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:21.690044 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.690333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.690616 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] PATH_INFO: `/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.691854 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6de0e48c1fe48ec8b4623540cff8eca)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:21.694091 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:21.709848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:21.740894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6de0e48c1fe48ec8b4623540cff8eca', 'request_id': 'req-ecc22181-6728-490f-b181-dd42a623ac12'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.747006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6de0e48c1fe48ec8b4623540cff8eca', 'request_id': 'req-ecc22181-6728-490f-b181-dd42a623ac12'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.752607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6de0e48c1fe48ec8b4623540cff8eca', 'request_id': 'req-ecc22181-6728-490f-b181-dd42a623ac12'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.764860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] The token cache is being invalidate because project b6de0e48c1fe48ec8b4623540cff8eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:21.765149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.765761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecc22181-6728-490f-b181-dd42a623ac12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:21.767321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1331/2662] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:21 2022] DELETE /identity/v3/projects/b6de0e48c1fe48ec8b4623540cff8eca => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:21.773100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.773572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.773675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.791386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:21.792583 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:21.849619 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1332/2663] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:21.858269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.858729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.858825 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.877848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:21.879045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:21.887400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1332/2664] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:21.891417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:21.902697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d7415-bffb-4e80-aba8-7ac8a6e60803 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:21.907331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:21.907988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.908282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.908567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.909309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:32:21.909622 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:21.911225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:21.917826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:21.925401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1333/2665] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:32:21.928446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:21.937572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1bc7e-fde4-4334-af6b-f7c7be1ad454 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:21.942659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:21.943347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:21.943637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:21.943915 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:21.944613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:21.946500 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:21.972346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1333/2666] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:21 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:21.976616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:21.988942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35804b0-429c-4e13-990b-9e188e9cf4d7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.046816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.047519 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.047802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.048084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.048773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.050667 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.074059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1334/2667] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:21 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:22.075291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.086269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8cf6d-bedc-46e2-bcc4-ea1afa6e2e76 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.090799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.091495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.091783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.092063 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.092838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.094417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.105116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1334/2668] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:32:22.109145 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.119843 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6929-c7b1-4a4a-b5c1-fb9966b12b04 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.124713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.125415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.125702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.125980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.126718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.128335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2557a2f-e0a8-4403-9d8c-408a06844a2d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.138892 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1335/2669] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:22.143050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.153217 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8fd9-5180-4eff-9940-7aba332e2a65 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.157790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.158675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.158962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.159265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] PATH_INFO: `/v3/projects/2ce26dfc8eac415c9433a08ad3f7b250/users/b803814f2f404a579a23848358b1c749/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.173416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce26dfc8eac415c9433a08ad3f7b250, user_id=b803814f2f404a579a23848358b1c749, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.176608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.189286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1335/2670] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:22 2022] PUT /identity/v3/projects/2ce26dfc8eac415c9433a08ad3f7b250/users/b803814f2f404a579a23848358b1c749/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:22.199949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.200244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.200522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.226425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] MFA Rules not processed for user `b803814f2f404a579a23848358b1c749`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:22.227680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:22.292772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1336/2671] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:22.297726 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.363999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412cdc9-0338-40b1-8322-5eb8bb7fd2e5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.368591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.369261 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.369550 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.369831 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.370527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.372539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.406292 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1336/2672] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:22 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:22.410456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.422360 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7249bc-a8f2-4a8d-843a-e889b393b9e6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.484355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.485048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.485336 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.485616 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.486323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.488298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.499447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:22.511582 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1337/2673] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:22 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:22.515553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.526989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e8260-6802-4938-9288-e7f70f6e8240 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.531680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.532452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.532763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.533042 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.533781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.535918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.547877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1337/2674] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:32:22.552569 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.563164 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1accd-81ff-42f5-acb5-de4efbdf93be admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.567950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.568903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.569852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.569852 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] PATH_INFO: `/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3/users/7df117b469f746408409dd6c3b8ad497/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.587144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38971fa3a1264fb0b0a91f66ab0226e3, user_id=7df117b469f746408409dd6c3b8ad497, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.589802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.607677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1338/2675] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:22 2022] PUT /identity/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3/users/7df117b469f746408409dd6c3b8ad497/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:22.611645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.623586 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a980d1-7e16-4ab3-9398-66e8b2e4b2d0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.632827 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:22.687083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.688177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.688489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.688772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.689686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.691419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe84f-710e-46cf-993b-216f0b31afab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.701753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1338/2676] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:22.705909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.716776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7809e-c767-4a02-9b4e-594e6cd890a5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.721773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.722682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.723517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.723800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] PATH_INFO: `/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3/users/7df117b469f746408409dd6c3b8ad497/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.725789 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38971fa3a1264fb0b0a91f66ab0226e3, user_id=7df117b469f746408409dd6c3b8ad497, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.728468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.740503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1339/2677] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:22 2022] PUT /identity/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3/users/7df117b469f746408409dd6c3b8ad497/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:22.750861 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.751152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.751463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.778538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] MFA Rules not processed for user `7df117b469f746408409dd6c3b8ad497`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:22.779785 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:22.853354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1339/2678] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:22.864379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:22.866205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:22.926442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1fa6-d1ee-46be-b088-35f036b139c1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:22.930028 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:22.930681 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:22.930965 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:22.931352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:22.940417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:22.941888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:22.951458 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1340/2679] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:23.097487 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:23.107960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16832913-a5c7-4b3a-af96-6ecb38c89f32 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:23.111490 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:23.112135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:23.112422 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:23.112701 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:23.182877 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:23.184268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1e44e-35da-4ac2-9ea6-e6de930c945b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:23.193596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1340/2680] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:30.446580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:30.507510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a22b2-0b01-410a-8a77-b7e55a7e47b2 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:30.511870 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:30.512732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:30.513013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:30.513288 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] PATH_INFO: `/v3/users/b803814f2f404a579a23848358b1c749` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:30.514447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b803814f2f404a579a23848358b1c749)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:30.516417 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:30.561665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b803814f2f404a579a23848358b1c749'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:30.562302 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b803814f2f404a579a23848358b1c749'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:30.570068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b803814f2f404a579a23848358b1c749'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:30.577125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:30.583425 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b803814f2f404a579a23848358b1c749'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:30.591069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b803814f2f404a579a23848358b1c749'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:30.597013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1341/2681] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:30 2022] DELETE /identity/v3/users/b803814f2f404a579a23848358b1c749 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:30.601510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:30.601967 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:30.602078 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:30.620142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:30.621332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:30.651559 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:30.682912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1341/2682] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:30.693072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:30.694523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:30.751252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0200a57-4ccf-4b2c-8843-c2d542e2dc85 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:30.755577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:30.756062 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:30.756353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:30.756636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:30.768575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:30.770392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:30.782461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1342/2683] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:31.123367 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:31.124813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:31.138187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d0e1e-efa3-43f2-b1cc-b7c1129f398a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:31.142812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:31.143719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.144007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.144812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] PATH_INFO: `/v3/projects/2ce26dfc8eac415c9433a08ad3f7b250` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.145980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce26dfc8eac415c9433a08ad3f7b250)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:31.148187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:31.194232 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce26dfc8eac415c9433a08ad3f7b250', 'request_id': 'req-9d97badc-6a21-414d-98de-04c2e63af2b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.200320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce26dfc8eac415c9433a08ad3f7b250', 'request_id': 'req-9d97badc-6a21-414d-98de-04c2e63af2b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.205840 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce26dfc8eac415c9433a08ad3f7b250', 'request_id': 'req-9d97badc-6a21-414d-98de-04c2e63af2b5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.219047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] The token cache is being invalidate because project 2ce26dfc8eac415c9433a08ad3f7b250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:31.219371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.219979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.221509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1342/2684] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:31 2022] DELETE /identity/v3/projects/2ce26dfc8eac415c9433a08ad3f7b250 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:31.226130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:31.235728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bee7d6-7fd4-442c-bfa8-92709ee721ef service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:31.291565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:31.292441 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.292727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.293034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] PATH_INFO: `/v3/users/7df117b469f746408409dd6c3b8ad497` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.294211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df117b469f746408409dd6c3b8ad497)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:31.296169 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:31.341605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df117b469f746408409dd6c3b8ad497'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.342182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df117b469f746408409dd6c3b8ad497'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.349647 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df117b469f746408409dd6c3b8ad497'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.361606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df117b469f746408409dd6c3b8ad497'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.368722 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6d3bf1b6-6dd3-40a1-8d33-059d6444ea70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df117b469f746408409dd6c3b8ad497'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.374717 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1343/2685] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:31 2022] DELETE /identity/v3/users/7df117b469f746408409dd6c3b8ad497 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:31.636116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:31.637560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:31.651833 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97badc-6a21-414d-98de-04c2e63af2b5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:31.711061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:31.711948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.712238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.712515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] PATH_INFO: `/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.713677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38971fa3a1264fb0b0a91f66ab0226e3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:31.715603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:31.759043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38971fa3a1264fb0b0a91f66ab0226e3', 'request_id': 'req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.764856 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38971fa3a1264fb0b0a91f66ab0226e3', 'request_id': 'req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.768626 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:31.771599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38971fa3a1264fb0b0a91f66ab0226e3', 'request_id': 'req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.788320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] The token cache is being invalidate because project 38971fa3a1264fb0b0a91f66ab0226e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:31.788617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.789235 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fa53836b-3ccf-4eb8-9cc7-dffe8aea24d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:31.791199 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1343/2686] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:31 2022] DELETE /identity/v3/projects/38971fa3a1264fb0b0a91f66ab0226e3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:31.795951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.796404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.796518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.802836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:31.817139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:31.818364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:31.886820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1344/2687] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:31.896098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.896390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.896682 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.919273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:31.920652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:31.934311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1344/2688] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:31.935439 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:31.946573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06bf17-d5e5-4dd5-aa53-8498523269d1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:31.951199 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:31.951889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.952176 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.952453 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.953184 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:32:31.953493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:31.955099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:31.970155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1345/2689] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:32:31.971139 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:31.980687 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7adcb-270a-4ef8-9f28-beed0ac28c9f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:31.986067 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:31.986734 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:31.987029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:31.987337 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:31.988566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:31.990466 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.025365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1345/2690] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:31 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:32.026798 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.039942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c76101-2b30-42cd-ad44-6c5b00b5ba56 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.121230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.122974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.123485 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.123966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.124884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.127493 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.165732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1346/2691] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:32.169757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.182403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e43a35-4944-4e0c-bf8d-d84c129b240d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.187532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.188283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.188645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.188986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.189813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.192874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.207999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1346/2692] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:32.218067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.229753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebc8-9995-4ac1-81df-8a6f4132d3de admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.234592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.235342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.235576 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.235884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.236657 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.253216 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d929b6-8621-487b-801b-6a4dd5c431e2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.264359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1347/2693] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:32.268644 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.279714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278e2bd-ab7e-480d-b386-c3df04bfe24c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.284356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.285354 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.285641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.285920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] PATH_INFO: `/v3/projects/29f0c3219807488784b24cbcdc49309c/users/35b8a1edd7a943be8d1999bbde53dec2/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.301801 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=29f0c3219807488784b24cbcdc49309c, user_id=35b8a1edd7a943be8d1999bbde53dec2, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.304467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.317916 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1347/2694] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:32 2022] PUT /identity/v3/projects/29f0c3219807488784b24cbcdc49309c/users/35b8a1edd7a943be8d1999bbde53dec2/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:32.327937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.328236 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.328520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.354862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] MFA Rules not processed for user `35b8a1edd7a943be8d1999bbde53dec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:32.356077 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:32.414891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:32.420871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1348/2695] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:32.424902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.492677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb7069-b3c3-4fb8-a364-505309e1d0ff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.497745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.498636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.498790 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.499121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.499859 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.501783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.533609 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1348/2696] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:32.537548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.548560 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc6dff-5f8d-402f-9d4c-865a07d3c406 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.615999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.617375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.617752 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.617985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.619125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.621405 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.644655 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1349/2697] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:32.648735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.662248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbaf7c-b944-4edd-a720-55538b8c256f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.667608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.668758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.669214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.669871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.670358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.672776 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.684768 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1349/2698] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:32.689068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.699701 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcaa9-cd64-4553-822f-291947d7f001 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.704479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.705144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.705430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.705711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.706448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.708100 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cddf55-6797-47dc-a6ec-649e4287f299 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.718287 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1350/2699] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:32.722324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.732476 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2866c8-ac90-4039-b767-7d32272e2384 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.737204 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.738604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.738894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.739176 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] PATH_INFO: `/v3/projects/35034c4c8a8b4b3f99126cbfb7ef2230/users/b4973f3730414361b2c1b223836fc73f/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.754465 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35034c4c8a8b4b3f99126cbfb7ef2230, user_id=b4973f3730414361b2c1b223836fc73f, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.757156 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.769437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1350/2700] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:32 2022] PUT /identity/v3/projects/35034c4c8a8b4b3f99126cbfb7ef2230/users/b4973f3730414361b2c1b223836fc73f/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:32.780753 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.781076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.781361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.807322 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] MFA Rules not processed for user `b4973f3730414361b2c1b223836fc73f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:32.808523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:32.871492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1351/2701] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:32.875758 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.909484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:32.937754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe35d0-101f-4e5f-be99-fd5d32adbf98 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:32.942830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:32.943527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:32.943821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:32.944102 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:32.944803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:32.946698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:32.975781 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1351/2702] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:32.976746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:32.987642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea523a7-7938-45df-a508-3fcbcf66b60c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.047525 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.048326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.048620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.048900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.049630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.052373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.062766 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:33.077404 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1352/2703] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:32 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:33.078793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.089220 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464a53d-2c74-4d21-9278-9bd23b41d0c0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.093991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.094727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.095094 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.095457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.096195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.098994 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.113848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1352/2704] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:33.115375 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.125606 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55afe92-8acd-4086-9508-2a6f9e3f8f45 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.130423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.131351 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.131635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.131912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] PATH_INFO: `/v3/projects/dc03aab7da6542059e7e9482f1198d07/users/8e7f495d6dde4bc587294b5110b44198/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.146515 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc03aab7da6542059e7e9482f1198d07, user_id=8e7f495d6dde4bc587294b5110b44198, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.149442 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.163863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1353/2705] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:33 2022] PUT /identity/v3/projects/dc03aab7da6542059e7e9482f1198d07/users/8e7f495d6dde4bc587294b5110b44198/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:33.167688 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.178477 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fbf2e-7298-408a-9fe8-d2c8cffb2f3c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.237507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.238777 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.239061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.239390 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.241518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.242787 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f980e144-da21-4228-84c9-8a9168cd357f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.254182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1353/2706] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:33.258361 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.268958 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69007d-845a-4829-ba8d-e554ab76dc16 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.274084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.275006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.275320 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.275601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] PATH_INFO: `/v3/projects/dc03aab7da6542059e7e9482f1198d07/users/8e7f495d6dde4bc587294b5110b44198/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.277573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc03aab7da6542059e7e9482f1198d07, user_id=8e7f495d6dde4bc587294b5110b44198, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.280248 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.292088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1354/2707] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:33 2022] PUT /identity/v3/projects/dc03aab7da6542059e7e9482f1198d07/users/8e7f495d6dde4bc587294b5110b44198/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:33.355860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d62f3-48b0-47b6-baff-449416ee767a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.356154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d62f3-48b0-47b6-baff-449416ee767a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.356437 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d62f3-48b0-47b6-baff-449416ee767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.382937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f21d62f3-48b0-47b6-baff-449416ee767a None None] MFA Rules not processed for user `8e7f495d6dde4bc587294b5110b44198`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:33.384175 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f21d62f3-48b0-47b6-baff-449416ee767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:33.452398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1354/2708] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:33.463999 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.465707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:33.522820 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89222bf-00e7-44f2-b490-4de0929b90c1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.526379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.527023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.527347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.527633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.581465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.583424 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.592160 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1355/2709] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:33.734315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.734850 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.735333 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.754713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:33.756008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:33.781767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:33.817298 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1355/2710] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:32:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:33.836621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:33.838048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:33.933310 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35664650-cd46-4678-ac99-a44e15d82eab service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:33.939737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:33.940773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:33.941602 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:33.941888 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:33.950684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:33.952318 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:33.961113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1356/2711] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:32:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:32:35.086433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:35.087852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:35.099653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7841e48-54ae-475d-b27d-cc80140a3913 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:35.104056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:35.104786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:35.105043 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:35.105351 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:35.160622 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:35.162545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:35.171285 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1356/2712] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:35.694985 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:35.699325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:35.705658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c8c14-5187-43c1-a357-83b0583ff5a3 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:35.709400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:35.710042 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:35.710325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:35.710600 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:35.720279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:35.721704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84eef4-bf6f-4287-bf9e-ce475f253c7e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:35.730926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1357/2713] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:39.582983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:39.643804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c25d-d446-43a0-9824-f2ae5db1bd0f service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:39.648544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:39.649403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:39.649686 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:39.649961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] PATH_INFO: `/v3/users/35b8a1edd7a943be8d1999bbde53dec2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:39.651122 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b8a1edd7a943be8d1999bbde53dec2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:39.653054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:39.697918 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b8a1edd7a943be8d1999bbde53dec2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:39.709526 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b8a1edd7a943be8d1999bbde53dec2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:39.717653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b8a1edd7a943be8d1999bbde53dec2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:39.719811 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:39.726153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b8a1edd7a943be8d1999bbde53dec2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:39.727587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b8a1edd7a943be8d1999bbde53dec2'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:39.734229 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1357/2714] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:39 2022] DELETE /identity/v3/users/35b8a1edd7a943be8d1999bbde53dec2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:39.738950 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:39.739469 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:39.739593 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:39.757558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:39.758755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:39.816050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1358/2715] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:39.825463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:39.826866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:39.884085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbabfb6-5660-49e9-9028-3e8ab1877240 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:39.887732 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:39.888380 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:39.888669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:39.888953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:39.898471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:39.899948 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:39.909398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1358/2716] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:40.175899 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:40.177362 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:40.191739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cff24-43bd-44aa-88e4-f593fc14f0d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:40.197455 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:40.198579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:40.198869 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:40.199158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] PATH_INFO: `/v3/projects/29f0c3219807488784b24cbcdc49309c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:40.200450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29f0c3219807488784b24cbcdc49309c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:40.202314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:40.250822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29f0c3219807488784b24cbcdc49309c', 'request_id': 'req-b8e4069e-78d7-4868-902c-bda4e5f3ab66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.258671 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29f0c3219807488784b24cbcdc49309c', 'request_id': 'req-b8e4069e-78d7-4868-902c-bda4e5f3ab66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.264891 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29f0c3219807488784b24cbcdc49309c', 'request_id': 'req-b8e4069e-78d7-4868-902c-bda4e5f3ab66'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.275697 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] The token cache is being invalidate because project 29f0c3219807488784b24cbcdc49309c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:40.275984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.276601 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.278143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1359/2717] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:40 2022] DELETE /identity/v3/projects/29f0c3219807488784b24cbcdc49309c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:40.282103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:40.292315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf7f2f-5900-487f-beee-b594ffdb83df service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:40.347433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:40.348240 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:40.348527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:40.348809 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] PATH_INFO: `/v3/users/b4973f3730414361b2c1b223836fc73f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:40.349998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4973f3730414361b2c1b223836fc73f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:40.352594 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:40.398223 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4973f3730414361b2c1b223836fc73f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.410533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4973f3730414361b2c1b223836fc73f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.417891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4973f3730414361b2c1b223836fc73f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.424743 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4973f3730414361b2c1b223836fc73f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.425373 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4973f3730414361b2c1b223836fc73f'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.431112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1359/2718] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:40 2022] DELETE /identity/v3/users/b4973f3730414361b2c1b223836fc73f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:40.691914 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:40.693418 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:40.706747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4069e-78d7-4868-902c-bda4e5f3ab66 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:40.764078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:40.764945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:40.765228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:40.765506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] PATH_INFO: `/v3/projects/35034c4c8a8b4b3f99126cbfb7ef2230` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:40.766771 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35034c4c8a8b4b3f99126cbfb7ef2230)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:40.768745 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:40.784408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:40.815291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35034c4c8a8b4b3f99126cbfb7ef2230', 'request_id': 'req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.822621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35034c4c8a8b4b3f99126cbfb7ef2230', 'request_id': 'req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.828768 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35034c4c8a8b4b3f99126cbfb7ef2230', 'request_id': 'req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.839205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] The token cache is being invalidate because project 35034c4c8a8b4b3f99126cbfb7ef2230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:40.839526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.840163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.841684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1360/2719] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:40 2022] DELETE /identity/v3/projects/35034c4c8a8b4b3f99126cbfb7ef2230 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:40.845794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:40.847391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:40.860034 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dcc94-87cb-43c4-8876-afb8b7ef90e3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:40.869434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:40.917224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:40.918110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:40.918395 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:40.918680 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] PATH_INFO: `/v3/users/8e7f495d6dde4bc587294b5110b44198` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:40.919874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7f495d6dde4bc587294b5110b44198)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:40.921784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:40.967470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7f495d6dde4bc587294b5110b44198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.980404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7f495d6dde4bc587294b5110b44198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.987760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7f495d6dde4bc587294b5110b44198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.995041 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7f495d6dde4bc587294b5110b44198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:40.995691 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ca9dad84-ea1b-4be8-a878-b1e1a830885b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7f495d6dde4bc587294b5110b44198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.001524 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1360/2720] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:32:40 2022] DELETE /identity/v3/users/8e7f495d6dde4bc587294b5110b44198 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:41.276366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.277827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:41.298550 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97490-b8f8-4ed7-a5c4-73eb0c042c95 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.355190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.356093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.356383 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.356665 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] PATH_INFO: `/v3/projects/dc03aab7da6542059e7e9482f1198d07` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.357871 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc03aab7da6542059e7e9482f1198d07)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.359837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.407153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc03aab7da6542059e7e9482f1198d07', 'request_id': 'req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.414947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc03aab7da6542059e7e9482f1198d07', 'request_id': 'req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.421219 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc03aab7da6542059e7e9482f1198d07', 'request_id': 'req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.431920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] The token cache is being invalidate because project dc03aab7da6542059e7e9482f1198d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:32:41.432207 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.432831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25b9c298-ee4f-4bf9-9103-2c5b6724e0aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:32:41.434352 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1361/2721] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:32:41 2022] DELETE /identity/v3/projects/dc03aab7da6542059e7e9482f1198d07 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:41.440822 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.441113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.441397 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.459182 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:41.460418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:41.518564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1361/2722] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:41.526894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.527190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.527507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.546008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:41.547267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:41.555151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1362/2723] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:41.559092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.568531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeb72e-7b4d-4578-9aa5-819b328894be None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.573012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.573658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.573945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.574222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.574951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:32:41.575284 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.576902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.588619 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1362/2724] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:32:41.592730 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.602463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cd7dc-ec41-4c61-a4c8-3a4cfeaaade3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.607158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.607834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.608123 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.608403 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.609089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.610973 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.635649 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1363/2725] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:41 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:41.639548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.650378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d62f9-6785-4003-bd7d-0c576bf97278 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.667816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:41.707621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.708288 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.708574 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.708851 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.709531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.711982 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.731646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1363/2726] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:41 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:41.735625 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.742400 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:41.746522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe342d-0171-40e1-8181-053acbbe9f14 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.752564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.753257 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.753546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.753825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.754551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.756178 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.766349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1364/2727] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:41.770112 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.780788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c02e5-e153-4ce7-90b0-ce3b5e8c7c28 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.785419 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.786080 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.786371 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.786660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.787446 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.789058 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6bb7-85f5-40b4-aa3a-2b85fc0547b5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.799812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1364/2728] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:41.803516 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.813289 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa65ee8-db78-4f45-86a5-eee8550e450b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:41.817854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:41.818726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.819012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.819323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] PATH_INFO: `/v3/projects/6d724d364f9b4bafb241a0bb0b02429f/users/7f9e0665a37d490f8b62205c5890da56/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.833949 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d724d364f9b4bafb241a0bb0b02429f, user_id=7f9e0665a37d490f8b62205c5890da56, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:41.837154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:41.849572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1365/2729] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:41 2022] PUT /identity/v3/projects/6d724d364f9b4bafb241a0bb0b02429f/users/7f9e0665a37d490f8b62205c5890da56/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:41.855418 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e637b4-3be8-4868-a244-2730c28d041f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:41.855889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e637b4-3be8-4868-a244-2730c28d041f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:41.856023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e637b4-3be8-4868-a244-2730c28d041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:41.873321 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d2e637b4-3be8-4868-a244-2730c28d041f None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:41.874541 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d2e637b4-3be8-4868-a244-2730c28d041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:41.933356 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1365/2730] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:41.942909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:41.944645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:42.001018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316cf86-9576-482f-a7ca-70dbd4b32da6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:42.004645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:42.005294 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:42.005584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:42.005866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:42.016234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:42.017889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:42.028347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1366/2731] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:45.935868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:45.935868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:45.935868 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:45.963812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] MFA Rules not processed for user `7f9e0665a37d490f8b62205c5890da56`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:45.965052 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:46.030098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1366/2732] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:46.034552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.045059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a58e-4824-4899-9954-3ce7dc08fdb5 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.049639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.050304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.050586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.050865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.051624 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.053577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.080780 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1367/2733] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:32:46 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:32:46.084547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.100917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bcf58-5809-43b3-9f88-130b7ba5615b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.142869 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:46.185049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.185907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.186198 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.186495 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.187349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.189294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.210533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1367/2734] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:32:46 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:32:46.214942 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.225813 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f678d4-3bf4-4f57-83e7-3525809304e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.236988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.237731 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.238032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.238321 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.242353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.242353 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.254719 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1368/2735] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:46.258343 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.269051 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2665ac-c9f3-4784-8a85-b4d67c35d08f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.276543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.278315 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.278675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.279024 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] PATH_INFO: `/v3/projects/260c02062b8247ee8adff900243d628a/users/34c23d5790054071b9326be4a7c3036b/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.296662 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260c02062b8247ee8adff900243d628a, user_id=34c23d5790054071b9326be4a7c3036b, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.299786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.315947 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1368/2736] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:46 2022] PUT /identity/v3/projects/260c02062b8247ee8adff900243d628a/users/34c23d5790054071b9326be4a7c3036b/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:32:46.319905 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.330484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39682c-ef43-40e3-9802-47308755e82f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.401096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.401896 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.402210 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.402489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.403258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.405607 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.419296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1369/2737] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:32:46.423152 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:46.434274 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e5720-8d4a-4724-933b-0e94e0706dc9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:46.440523 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:46.441433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:46.441717 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:46.441998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] PATH_INFO: `/v3/projects/260c02062b8247ee8adff900243d628a/users/34c23d5790054071b9326be4a7c3036b/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:46.444039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=260c02062b8247ee8adff900243d628a, user_id=34c23d5790054071b9326be4a7c3036b, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:46.447328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cecd98-83a7-4880-b4b1-a958f7253175 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:46.460307 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1369/2738] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:32:46 2022] PUT /identity/v3/projects/260c02062b8247ee8adff900243d628a/users/34c23d5790054071b9326be4a7c3036b/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:32:48.730300 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:48.735680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:48.812143 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:32:48.856487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b527783-9c5f-4321-974b-1e732131e820 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:48.968333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:48.969249 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:48.969677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:48.970135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:48.972242 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:48.997350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:48.999132 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:49.012209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1370/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:32:50.973346 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cb53-3699-4d30-a4fa-363457e5c5d9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:50.973875 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cb53-3699-4d30-a4fa-363457e5c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:50.974362 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cb53-3699-4d30-a4fa-363457e5c5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:51.024045 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c585cb53-3699-4d30-a4fa-363457e5c5d9 None None] MFA Rules not processed for user `34c23d5790054071b9326be4a7c3036b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:32:51.025439 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c585cb53-3699-4d30-a4fa-363457e5c5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:51.127264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1370/2740] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:32:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:32:51.141920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:32:51.145089 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:32:51.255656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33aea3-8395-439b-a831-75d13ea34ccb service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:32:51.263194 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:32:51.264315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:32:51.264814 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:32:51.265270 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:32:51.348579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:32:51.350230 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ba4b-0a11-4be8-89a4-57c3dda18a82 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:32:51.379661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1371/2741] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:32:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:33:01.120596 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:33:01.357331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:33:07.586200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:33:07.586722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:33:07.586870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:33:07.604650 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] MFA Rules not processed for user `7f9e0665a37d490f8b62205c5890da56`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:33:07.605866 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:33:07.612779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1371/2742] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:33:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:33:17.598806 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:33:20.960483 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:33:20.960963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:33:20.961060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:33:20.980072 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] MFA Rules not processed for user `2f35de73cde9436c952e0d5f95e74b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:33:20.981273 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:33:21.035861 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1372/2743] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:33:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:33:22.647414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:33:22.648904 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:33:22.709180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06594a6-e444-490e-a6c7-7e0b6e4670fe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:33:22.713471 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:33:22.714263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:33:22.714551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:33:22.714830 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:33:22.726564 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:33:22.728766 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:33:22.743516 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1372/2744] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:33:31.029179 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:33:32.735767 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=26 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:01.573452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:01.583157 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d5063-1116-4c0e-a4cd-09dd86c9636b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:01.586931 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:01.587706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:01.588063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:01.588414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:01.590324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:01.653427 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:01.654902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:01.665991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1373/2745] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:34:06.203780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:06.205387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:06.226233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5696d7-45f3-4eb0-827a-72ffedf686c0 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:06.231359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:06.232177 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:06.232459 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:06.232735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] PATH_INFO: `/v3/users/f8ae63f0fa9e4c9ab82d46d6d24d22e3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:06.233958 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ae63f0fa9e4c9ab82d46d6d24d22e3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:06.235883 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:06.282618 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ae63f0fa9e4c9ab82d46d6d24d22e3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.294842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ae63f0fa9e4c9ab82d46d6d24d22e3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.301975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ae63f0fa9e4c9ab82d46d6d24d22e3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.309617 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ae63f0fa9e4c9ab82d46d6d24d22e3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.310249 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ae63f0fa9e4c9ab82d46d6d24d22e3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.316265 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1373/2746] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:34:06 2022] DELETE /identity/v3/users/f8ae63f0fa9e4c9ab82d46d6d24d22e3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:34:06.488006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:06.498838 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88554bc4-be47-4108-9cf9-cd5ad6f1ccc1 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:06.555790 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:06.556662 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:06.556954 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:06.557238 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] PATH_INFO: `/v3/projects/42279c5b622c47a192c529ce53566555` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:06.558454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=42279c5b622c47a192c529ce53566555)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:06.560384 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:06.606675 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42279c5b622c47a192c529ce53566555', 'request_id': 'req-fef49380-14e2-4b01-8366-6be71a466b2a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.613421 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42279c5b622c47a192c529ce53566555', 'request_id': 'req-fef49380-14e2-4b01-8366-6be71a466b2a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.619467 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42279c5b622c47a192c529ce53566555', 'request_id': 'req-fef49380-14e2-4b01-8366-6be71a466b2a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.629821 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] The token cache is being invalidate because project 42279c5b622c47a192c529ce53566555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:34:06.630113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.630743 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.632291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1374/2747] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:34:06 2022] DELETE /identity/v3/projects/42279c5b622c47a192c529ce53566555 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:34:06.636287 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:06.637754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:06.651404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420fb16-83ae-44f9-b931-5889b2e230bc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:06.705378 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:06.706247 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:06.706533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:06.706896 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] PATH_INFO: `/v3/users/1b49636b617b40708242bf30cf7db4a1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:06.708049 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b49636b617b40708242bf30cf7db4a1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:06.710029 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:06.755753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b49636b617b40708242bf30cf7db4a1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.767517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b49636b617b40708242bf30cf7db4a1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.775492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b49636b617b40708242bf30cf7db4a1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.777489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:06.784566 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b49636b617b40708242bf30cf7db4a1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.785436 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-71544f07-c6f8-4e1b-af46-f380a11428b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b49636b617b40708242bf30cf7db4a1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:06.792191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1374/2748] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:34:06 2022] DELETE /identity/v3/users/1b49636b617b40708242bf30cf7db4a1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:34:06.964911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:06.966542 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:06.979211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49380-14e2-4b01-8366-6be71a466b2a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:06.999146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:07.038480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:07.039380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:07.039689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:07.039970 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] PATH_INFO: `/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:07.041168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f8b24a40b54ec78fb9180d2f2ff24e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:07.043142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:07.087663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f8b24a40b54ec78fb9180d2f2ff24e', 'request_id': 'req-85028f25-51fd-40ef-9808-1455c1369814'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:07.095627 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f8b24a40b54ec78fb9180d2f2ff24e', 'request_id': 'req-85028f25-51fd-40ef-9808-1455c1369814'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:07.101759 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f8b24a40b54ec78fb9180d2f2ff24e', 'request_id': 'req-85028f25-51fd-40ef-9808-1455c1369814'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:07.112746 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] The token cache is being invalidate because project 57f8b24a40b54ec78fb9180d2f2ff24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:34:07.113043 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:07.113674 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:34:07.115200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1375/2749] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:34:06 2022] DELETE /identity/v3/projects/57f8b24a40b54ec78fb9180d2f2ff24e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:34:16.788020 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:17.109536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:30.146707 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:30.147279 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:30.147387 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:30.165448 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:34:30.166652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:30.222366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1375/2750] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:34:30.268020 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:30.269707 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:30.340015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85028f25-51fd-40ef-9808-1455c1369814 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:30.345742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:30.346487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:30.346677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:30.346955 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:30.355497 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:30.357084 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:30.365111 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1376/2751] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:34:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:34:37.040713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:37.042174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:37.116539 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d322098-c59a-4938-911d-fef8e712e900 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:37.120267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:37.120931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:37.121218 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:37.121545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:37.123423 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:37.213190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:37.214775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad8c7f-3126-46b9-94ea-27c8d5c3ecd4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:37.226909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1376/2752] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:34:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:34:38.430269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:38.432452 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:38.445612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0192e-d7c8-47c2-8477-3e0877ca1a28 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:38.450076 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:38.450758 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:38.451048 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:38.451420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:38.453473 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:38.470018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:38.471507 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:38.483325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1377/2753] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:34:44.521266 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619e32-1d28-4446-b4a7-7d1566f15f2d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:44.521730 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619e32-1d28-4446-b4a7-7d1566f15f2d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:44.521860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29619e32-1d28-4446-b4a7-7d1566f15f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:44.539452 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-29619e32-1d28-4446-b4a7-7d1566f15f2d None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:34:44.541153 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-29619e32-1d28-4446-b4a7-7d1566f15f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:44.548528 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1377/2754] 10.222.0.46 () {58 vars in 1009 bytes} [Wed Jul 6 15:34:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:34:44.560747 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:44.571862 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763aefd-e3df-41a6-9030-35f5f708d1fc service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:44.576840 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:44.577480 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:44.577770 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:44.578051 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:44.586155 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:44.587906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:44.597613 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1378/2755] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:34:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:34:54.534015 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=82 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:54.593228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=44 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:34:58.145057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:58.145561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:58.145684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:58.163140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] MFA Rules not processed for user `cbf595eb1a0c44afb4d4f96413763631`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:34:58.164873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:34:58.171986 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1378/2756] 10.222.0.46 () {58 vars in 967 bytes} [Wed Jul 6 15:34:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:34:58.184636 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:34:58.194936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9018-ae7e-456a-a9a1-94abb6468050 service swift] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:34:58.199926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44f05b05c1c94c44aa499f224ac20648', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:34:58.200563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:34:58.200847 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:34:58.201125 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:34:58.208967 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:34:58.210531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91571c33-4308-4533-a1e2-29ec0532c75b service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:34:58.218851 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1379/2757] 10.222.0.46 () {60 vars in 1384 bytes} [Wed Jul 6 15:34:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 15:35:08.158028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:08.215688 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:23.964570 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:23.966205 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:23.983841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73717-c0d4-40c0-a82c-2b4cd23b3c8b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:23.990391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:23.991136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:23.991694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:23.992821 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:23.994612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:24.012473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:24.014113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:24.039190 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1379/2758] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:24.835935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:24.836446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:24.836562 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:24.855274 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] MFA Rules not processed for user `7f9e0665a37d490f8b62205c5890da56`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:24.857344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:24.917581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1380/2759] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:27.282880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:27.292675 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38068-96c5-49e3-8b32-b8c877e86c62 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:27.297038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:27.297706 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:27.297996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:27.298275 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:27.300100 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:27.312097 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:27.313549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:27.322510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1380/2760] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:34.911251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:37.318536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=12 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:39.216706 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:39.227177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc398d3-d922-4af8-a949-de7af620e9ec None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:39.236459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:39.237513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:39.237991 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:39.238450 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:39.240513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:39.309279 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:39.311399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:39.321549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1381/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:42.329108 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:42.330576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:42.397652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:42.399987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:42.415920 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7addd-66fe-4fc3-ac48-551803e8d6b0 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:42.421342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:42.422240 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.422527 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.422807 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] PATH_INFO: `/v3/users/2f35de73cde9436c952e0d5f95e74b8b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.424221 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f35de73cde9436c952e0d5f95e74b8b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:42.425314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a712a6-c05e-437c-a5f4-b98d3ddf6584 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:42.426204 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:42.431061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:42.431860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.432155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.432587 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.434475 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:42.447871 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:42.449486 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:42.462404 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1381/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:42.485223 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f35de73cde9436c952e0d5f95e74b8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.485909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f35de73cde9436c952e0d5f95e74b8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.494033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f35de73cde9436c952e0d5f95e74b8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.506803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f35de73cde9436c952e0d5f95e74b8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.514440 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb82eb0f-e758-46e1-9c04-41348c7c2749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f35de73cde9436c952e0d5f95e74b8b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.520866 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1382/2763] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:35:42 2022] DELETE /identity/v3/users/2f35de73cde9436c952e0d5f95e74b8b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:35:42.708610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:42.720890 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93deca-7638-4ff6-8ec7-5d17d443320b service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:42.777623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:42.779087 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.779414 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.779695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] PATH_INFO: `/v3/projects/848d6c1f478d4dba813663c4c866d8b8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.780880 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=848d6c1f478d4dba813663c4c866d8b8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:42.782798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:42.828348 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848d6c1f478d4dba813663c4c866d8b8', 'request_id': 'req-33800455-1869-43ef-ad9f-8531b36f96ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.834295 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848d6c1f478d4dba813663c4c866d8b8', 'request_id': 'req-33800455-1869-43ef-ad9f-8531b36f96ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.839796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848d6c1f478d4dba813663c4c866d8b8', 'request_id': 'req-33800455-1869-43ef-ad9f-8531b36f96ab'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.852535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] The token cache is being invalidate because project 848d6c1f478d4dba813663c4c866d8b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:35:42.852839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.853468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-33800455-1869-43ef-ad9f-8531b36f96ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:42.855006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1382/2764] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:35:42 2022] DELETE /identity/v3/projects/848d6c1f478d4dba813663c4c866d8b8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:42.859951 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.860244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.860524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.878133 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:42.879392 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:42.937732 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1383/2765] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:42.946092 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.946389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.946676 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.966738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:42.967978 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:42.975874 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1383/2766] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:42.980016 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:42.989590 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1782e2-645a-4942-9afb-5c2e7f1a4098 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:42.994126 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:42.994777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:42.995060 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:42.995370 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:42.996099 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:35:42.996433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:42.998069 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:42.999548 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:43.012836 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1384/2767] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:35:43.017214 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.026813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a69d1-ce50-4c02-b66b-596dbf262a49 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.032056 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.032713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.033008 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.033299 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.033999 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.035970 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.045625 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:43.062621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1384/2768] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:35:43.066815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.077558 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d542f4c-b08c-4358-8885-a5550bc492ae admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.134117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.134728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.135018 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.135331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.136052 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.138545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.158163 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1385/2769] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:35:43.162381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.172571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7dd50-81f8-4790-a8b4-7f0a96272b77 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.177166 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.177829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.178118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.178401 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.179132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.180804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.191532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1385/2770] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:43.195332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.206054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20ae2b-9a21-4f2d-a92d-5f304584fe11 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.211175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.211865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.212181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.212492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.213217 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.214794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb4c36-c8ec-4fb2-b67b-3091e1bcb7df admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.224586 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1386/2771] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:43.228695 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.238489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867d91e-e31b-45de-b264-1c70484656e3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.245050 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.245938 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.246219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.246493 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] PATH_INFO: `/v3/projects/6ae1337c0fb34e4aba029a53f0b9c1ed/users/c4719449b4e0456b93576319d4c7b1f2/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.261984 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae1337c0fb34e4aba029a53f0b9c1ed, user_id=c4719449b4e0456b93576319d4c7b1f2, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.265410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.278360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1386/2772] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:43 2022] PUT /identity/v3/projects/6ae1337c0fb34e4aba029a53f0b9c1ed/users/c4719449b4e0456b93576319d4c7b1f2/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:43.288177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.288506 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.288791 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.315004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] MFA Rules not processed for user `c4719449b4e0456b93576319d4c7b1f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:43.316254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:43.379764 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1387/2773] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:43.384257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.443934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f81d5-5bfc-4922-8f7a-fc98ea27d5cd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.448531 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.449181 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.449464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.450324 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.451007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.452930 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.476746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1387/2774] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/projects => generated 363 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:35:43.480735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.491875 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ecb8-a360-44fd-91ce-c9203b9f836b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.547355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.548023 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.548308 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.548585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.549264 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.551159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.558498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:43.570668 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1388/2775] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:35:43.574643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.585310 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694b6f7-c5ba-4ec1-8993-bc63c4f1004f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.589995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.590664 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.590954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.591264 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.592004 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.593605 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.604083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1388/2776] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:35:43.608447 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.618222 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad028e58-28c8-4928-8e9a-59eda6c7d4b4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.623461 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.624349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.624639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.624921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] PATH_INFO: `/v3/projects/9e54180dd7564ef586014b59450fe3e2/users/a8d4a549168e4c28b201b46f329b6b42/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.639531 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e54180dd7564ef586014b59450fe3e2, user_id=a8d4a549168e4c28b201b46f329b6b42, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.642272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.655096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1389/2777] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:43 2022] PUT /identity/v3/projects/9e54180dd7564ef586014b59450fe3e2/users/a8d4a549168e4c28b201b46f329b6b42/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:43.659033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.669674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f2ea-998a-42c6-9263-44f8e3d797ff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.724038 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.724703 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.724988 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.725267 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.726018 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.727643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124606a9-9304-4874-a8ea-0dfb17547a34 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.737934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1389/2778] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:43.742196 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.753443 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3920a-e721-4bad-b9a6-cb4efd67d761 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.758049 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.758934 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.759241 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.759524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] PATH_INFO: `/v3/projects/9e54180dd7564ef586014b59450fe3e2/users/a8d4a549168e4c28b201b46f329b6b42/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.761478 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e54180dd7564ef586014b59450fe3e2, user_id=a8d4a549168e4c28b201b46f329b6b42, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.764788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.775134 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1390/2779] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:43 2022] PUT /identity/v3/projects/9e54180dd7564ef586014b59450fe3e2/users/a8d4a549168e4c28b201b46f329b6b42/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:43.786697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.787019 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.787329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.813728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] MFA Rules not processed for user `a8d4a549168e4c28b201b46f329b6b42`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:43.814937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:43.827965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:43.883857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1390/2780] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:43.895859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:43.897585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:43.954503 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366cacd-b29c-441a-a331-163a6313b5f4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:43.958117 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:43.958756 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:43.959039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:43.959345 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:43.968135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:43.969561 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:43.978761 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1391/2781] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:44.050533 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:44.061155 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691255c-4fa2-440e-b762-3e4216ff429c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:44.064775 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:44.065413 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:44.065702 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:44.065975 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:44.122497 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:44.123979 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224bcf9-e4d3-4e80-a4f5-4b77c029fc5a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:44.132642 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1391/2782] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:53.164205 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:53.223788 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9f27-7814-4f9d-8a4f-42b9dba81ac1 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:53.228286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:53.229154 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:53.229436 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:53.229710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] PATH_INFO: `/v3/users/c4719449b4e0456b93576319d4c7b1f2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:53.230879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4719449b4e0456b93576319d4c7b1f2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:53.233315 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:53.278611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4719449b4e0456b93576319d4c7b1f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.279245 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4719449b4e0456b93576319d4c7b1f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.286304 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4719449b4e0456b93576319d4c7b1f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.290650 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:53.301227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4719449b4e0456b93576319d4c7b1f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.307878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4719449b4e0456b93576319d4c7b1f2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.314347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1392/2783] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:35:53 2022] DELETE /identity/v3/users/c4719449b4e0456b93576319d4c7b1f2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:53.318923 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:53.319442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:53.319549 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:53.336728 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:53.337957 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:53.395489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1392/2784] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:35:53.405109 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:53.406536 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:53.463251 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed9223-3f2a-4836-b525-092fe43a2bb8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:53.466889 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:53.467566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:53.467855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:53.468139 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:53.477151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:53.478587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:53.488585 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1393/2785] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:35:53.749964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:53.751388 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:53.764449 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87582f96-34bf-4cd2-8e5e-f39138e5a235 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:53.769268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:53.770135 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:53.770912 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:53.771194 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] PATH_INFO: `/v3/projects/6ae1337c0fb34e4aba029a53f0b9c1ed` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:53.772581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae1337c0fb34e4aba029a53f0b9c1ed)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:53.774538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:53.817398 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:53.820548 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae1337c0fb34e4aba029a53f0b9c1ed', 'request_id': 'req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.827268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae1337c0fb34e4aba029a53f0b9c1ed', 'request_id': 'req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.832784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae1337c0fb34e4aba029a53f0b9c1ed', 'request_id': 'req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.845216 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] The token cache is being invalidate because project 6ae1337c0fb34e4aba029a53f0b9c1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:35:53.845503 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.846109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.847791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1393/2786] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:35:53 2022] DELETE /identity/v3/projects/6ae1337c0fb34e4aba029a53f0b9c1ed => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:53.851714 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:53.861974 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe452f4-1602-4050-9c27-7c39672f3b86 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:53.917151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:53.918050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:53.918340 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:53.918621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] PATH_INFO: `/v3/users/a8d4a549168e4c28b201b46f329b6b42` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:53.920510 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d4a549168e4c28b201b46f329b6b42)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:53.922459 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:53.966091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d4a549168e4c28b201b46f329b6b42'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.967292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d4a549168e4c28b201b46f329b6b42'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.974075 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d4a549168e4c28b201b46f329b6b42'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.985772 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d4a549168e4c28b201b46f329b6b42'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.992966 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0d862e33-81ae-4687-aa25-daee205679d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d4a549168e4c28b201b46f329b6b42'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:53.994781 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:54.000192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1394/2787] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:35:53 2022] DELETE /identity/v3/users/a8d4a549168e4c28b201b46f329b6b42 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:54.260224 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.261700 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:54.275482 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa51c5-28e1-4753-9531-14fbfa8b1f59 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.333542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.334410 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.334724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.335006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] PATH_INFO: `/v3/projects/9e54180dd7564ef586014b59450fe3e2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.336233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e54180dd7564ef586014b59450fe3e2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.338149 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.383690 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e54180dd7564ef586014b59450fe3e2', 'request_id': 'req-42514187-4f07-416b-977a-ba732ba0105c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:54.390464 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e54180dd7564ef586014b59450fe3e2', 'request_id': 'req-42514187-4f07-416b-977a-ba732ba0105c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:54.396421 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e54180dd7564ef586014b59450fe3e2', 'request_id': 'req-42514187-4f07-416b-977a-ba732ba0105c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:54.411331 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] The token cache is being invalidate because project 9e54180dd7564ef586014b59450fe3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:35:54.411780 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:54.412738 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-42514187-4f07-416b-977a-ba732ba0105c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:35:54.414328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1394/2788] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:35:54 2022] DELETE /identity/v3/projects/9e54180dd7564ef586014b59450fe3e2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:54.420033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.420544 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.420651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.439247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:54.440635 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:54.498587 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1395/2789] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:54.506355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.506760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.507047 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.525906 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:54.527114 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:54.534981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1395/2790] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:54.539012 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.548724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a52f8-daa0-4b43-8818-1f7f44c1879e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.553252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.553904 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.554193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.554470 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.555202 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:35:54.555543 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.557135 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.569355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1396/2791] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:35:54.573468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.583065 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f5870-f57c-4585-b68d-6263317d2545 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.587694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.588338 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.588629 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.588900 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.589600 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.591491 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.615762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1396/2792] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:35:54.619584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.630124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4c231-401e-40e9-9d0c-fe13ac83304a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.686347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.686947 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.687291 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.687534 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.688227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.690130 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.709693 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1397/2793] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:35:54.713713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.723828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20584cb7-dd66-4269-91b7-83757078ed01 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.728386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.729073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.729366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.729641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.730376 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.732517 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.742913 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1397/2794] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:54.746685 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.757144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadf95a-b1ac-4f04-bd8b-937febc836e9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.761757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.762411 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.762699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.763001 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.763802 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.765390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0179ced-8d49-4209-bebc-717d3e4258ee admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.775324 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1398/2795] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:54.779081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.788879 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597cb55-6517-4169-9777-d54008ae2551 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:54.793542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:54.794928 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.795211 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.795511 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] PATH_INFO: `/v3/projects/4d1fe32b01bc467e9a3434f0c5fd24ab/users/0b9d9ca6102c49c8a8c854811df3ab68/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.809659 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1fe32b01bc467e9a3434f0c5fd24ab, user_id=0b9d9ca6102c49c8a8c854811df3ab68, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:54.812903 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:54.821412 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:54.826956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1398/2796] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:54 2022] PUT /identity/v3/projects/4d1fe32b01bc467e9a3434f0c5fd24ab/users/0b9d9ca6102c49c8a8c854811df3ab68/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:54.837093 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:54.837386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:54.837704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:54.864116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] MFA Rules not processed for user `0b9d9ca6102c49c8a8c854811df3ab68`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:54.865325 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:54.918563 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:54.929855 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1399/2797] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:54.934303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:54.995365 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f1d7-152b-4c60-b1e0-36217ac80420 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.000563 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.001225 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.001509 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.001786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.002468 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.004391 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.028442 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1399/2798] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:35:54 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:35:55.033144 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.043713 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf66c-1f52-4b68-9f18-20eda73c5cc4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.097634 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.098297 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.098580 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.098858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.099584 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.101489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.121794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1400/2799] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:35:55 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:35:55.125893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.136163 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650bb1c-eb01-4175-9b60-703e2eb9814b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.141188 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.141848 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.142136 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.142669 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.143432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.145113 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.156046 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1400/2800] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:35:55.160008 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.170500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330b0d-a211-4b6b-b0e2-38440996f55c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.175088 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.175987 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.176271 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.176547 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] PATH_INFO: `/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38/users/8135bddd14b449d79d22d3eeb77f5606/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.190984 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8177c81a7fc443cdbbe46ba9e4731b38, user_id=8135bddd14b449d79d22d3eeb77f5606, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.193672 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.205990 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1401/2801] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:55 2022] PUT /identity/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38/users/8135bddd14b449d79d22d3eeb77f5606/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:35:55.209791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.219989 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a56f-e5a3-46a7-99d4-d513a53e45dc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.274649 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.275355 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.275641 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.275920 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.276646 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.278226 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d766e-3813-45f0-91fd-64b262552ee3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.288237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1401/2802] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 15:35:55.292494 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.303185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd982d80-4d20-4ade-8f8a-4592895ec938 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.307774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.308656 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.308940 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.309218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] PATH_INFO: `/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38/users/8135bddd14b449d79d22d3eeb77f5606/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.311203 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8177c81a7fc443cdbbe46ba9e4731b38, user_id=8135bddd14b449d79d22d3eeb77f5606, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.313825 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.323902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1402/2803] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:35:55 2022] PUT /identity/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38/users/8135bddd14b449d79d22d3eeb77f5606/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:35:55.335086 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481c14a-df38-456d-83a7-19163830061c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.335631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481c14a-df38-456d-83a7-19163830061c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.335737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481c14a-df38-456d-83a7-19163830061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.362845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c481c14a-df38-456d-83a7-19163830061c None None] MFA Rules not processed for user `8135bddd14b449d79d22d3eeb77f5606`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:35:55.364110 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c481c14a-df38-456d-83a7-19163830061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:55.421991 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:55.431696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1402/2804] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:35:55.442663 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:35:55.444489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:35:55.501464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c51ba-4d6c-4742-a98c-d3080b69e86a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:35:55.505200 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:35:55.505793 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:35:55.506110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:35:55.506393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:35:55.563908 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:35:55.565379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:35:55.569900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:35:55.575170 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1403/2805] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:36:00.154917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481c14a-df38-456d-83a7-19163830061c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:00.165236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481c14a-df38-456d-83a7-19163830061c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:00.168789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:00.169463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:00.169747 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:00.170028 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:00.179001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:00.180441 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e1768-26c1-444c-acba-166b1ff42fc5 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:00.189647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1403/2806] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:10.185540 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:20.756228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:20.816762 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e2a28-725c-4d3b-8094-dacecf73cc70 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:20.821116 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:20.821975 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:20.822259 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:20.822532 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] PATH_INFO: `/v3/users/0b9d9ca6102c49c8a8c854811df3ab68` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:20.823715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9d9ca6102c49c8a8c854811df3ab68)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:20.825621 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:20.869704 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9d9ca6102c49c8a8c854811df3ab68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:20.870342 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9d9ca6102c49c8a8c854811df3ab68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:20.877784 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9d9ca6102c49c8a8c854811df3ab68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:20.889487 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9d9ca6102c49c8a8c854811df3ab68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:20.896529 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9d9ca6102c49c8a8c854811df3ab68'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:20.902335 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1404/2807] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:20 2022] DELETE /identity/v3/users/0b9d9ca6102c49c8a8c854811df3ab68 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:20.906804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:20.907286 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:20.907432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:20.925578 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:20.926778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:20.983151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1404/2808] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:20.993254 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:20.994774 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:21.050983 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7458e-2fe9-46bd-b8ad-62e1173c7474 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:21.054682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:21.055357 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:21.055652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:21.055930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:21.065149 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:21.066577 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:21.076032 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1405/2809] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:21.339447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:21.340956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:21.354036 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28d4f-ccb7-4b8e-adb3-f4eb01224887 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:21.358572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:21.359557 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:21.359845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:21.360125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] PATH_INFO: `/v3/projects/4d1fe32b01bc467e9a3434f0c5fd24ab` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:21.361314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1fe32b01bc467e9a3434f0c5fd24ab)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:21.363248 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:21.410215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1fe32b01bc467e9a3434f0c5fd24ab', 'request_id': 'req-c7dde6f7-966f-4a1a-a9be-bef125ab539c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.416335 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1fe32b01bc467e9a3434f0c5fd24ab', 'request_id': 'req-c7dde6f7-966f-4a1a-a9be-bef125ab539c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.421710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1fe32b01bc467e9a3434f0c5fd24ab', 'request_id': 'req-c7dde6f7-966f-4a1a-a9be-bef125ab539c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.433603 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] The token cache is being invalidate because project 4d1fe32b01bc467e9a3434f0c5fd24ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:21.433893 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.434507 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.436062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1405/2810] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:21 2022] DELETE /identity/v3/projects/4d1fe32b01bc467e9a3434f0c5fd24ab => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:21.440153 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:21.449538 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4ac3-8ae2-4b25-8a83-fd8126a0c2e5 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:21.467651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:21.507410 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:21.508282 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:21.508568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:21.508845 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] PATH_INFO: `/v3/users/8135bddd14b449d79d22d3eeb77f5606` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:21.510033 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8135bddd14b449d79d22d3eeb77f5606)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:21.511977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:21.557323 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8135bddd14b449d79d22d3eeb77f5606'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.557977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8135bddd14b449d79d22d3eeb77f5606'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.564227 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8135bddd14b449d79d22d3eeb77f5606'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.575830 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8135bddd14b449d79d22d3eeb77f5606'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.583119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-650a153c-7329-4be5-9e94-11be146c85ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8135bddd14b449d79d22d3eeb77f5606'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.588912 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1406/2811] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:21 2022] DELETE /identity/v3/users/8135bddd14b449d79d22d3eeb77f5606 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:21.853246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:21.854729 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:21.872935 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde6f7-966f-4a1a-a9be-bef125ab539c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:21.929535 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:21.930432 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:21.930735 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:21.931012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] PATH_INFO: `/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:21.932236 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8177c81a7fc443cdbbe46ba9e4731b38)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:21.934137 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:21.963910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:21.980704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8177c81a7fc443cdbbe46ba9e4731b38', 'request_id': 'req-a29560fe-bf56-4071-92f2-faa595e0b03c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.987128 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8177c81a7fc443cdbbe46ba9e4731b38', 'request_id': 'req-a29560fe-bf56-4071-92f2-faa595e0b03c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:21.992537 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8177c81a7fc443cdbbe46ba9e4731b38', 'request_id': 'req-a29560fe-bf56-4071-92f2-faa595e0b03c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:22.004716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] The token cache is being invalidate because project 8177c81a7fc443cdbbe46ba9e4731b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:22.005098 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:22.005727 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a29560fe-bf56-4071-92f2-faa595e0b03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:22.007259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1406/2812] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:21 2022] DELETE /identity/v3/projects/8177c81a7fc443cdbbe46ba9e4731b38 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:22.012105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.012578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.012676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.030495 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:22.031728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:22.089874 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1407/2813] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:22.097518 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.097759 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.098037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.116289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:22.117515 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:22.128812 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1407/2814] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:22.131884 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.141605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85084-cb53-4d77-97b8-01e882c09c16 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.146549 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.147231 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.147524 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.147805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.148545 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:36:22.148858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.150475 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.162692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1408/2815] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:36:22.167145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.176754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c0c3b-5fb6-4884-b079-3a5b5dda9253 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.181358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.182005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.182289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.182562 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.183280 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.185143 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.209660 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1408/2816] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:22.213828 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.223724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad893264-efda-4f83-8b62-e00e57b1da68 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.249378 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:22.283386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.284092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.284380 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.284702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.285398 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.287364 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.307740 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1409/2817] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:22.310589 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.321250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e8c16-e763-4741-a556-c6303959edbd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.325799 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.326457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.326746 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.327023 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.327793 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.329369 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.339571 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1409/2818] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:22.343312 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.353903 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e25796-39d2-4115-959f-1cebf82403ea admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.358462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.359124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.359451 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.359734 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.360492 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.362718 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01d0a0-2f37-4476-8c86-4f0eff249389 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.372677 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1410/2819] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:22.376327 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.386271 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170b46-58f1-4190-8cd8-86c87d8c5409 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.390763 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.391668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.391953 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.392230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] PATH_INFO: `/v3/projects/29fc48878a414050b14d0ddfd451cb5a/users/a4d1fed6ddd84dc2907336cc18adea1b/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.406366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fc48878a414050b14d0ddfd451cb5a, user_id=a4d1fed6ddd84dc2907336cc18adea1b, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.409668 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.421910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1410/2820] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:22 2022] PUT /identity/v3/projects/29fc48878a414050b14d0ddfd451cb5a/users/a4d1fed6ddd84dc2907336cc18adea1b/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:22.482517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.482809 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.483098 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.509309 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] MFA Rules not processed for user `a4d1fed6ddd84dc2907336cc18adea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:22.510513 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:22.573739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1411/2821] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:22.578131 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.642005 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5919d1-9c22-4e2f-bae6-e2c81ed8b573 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.646945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.647674 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.647962 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.648242 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.648933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.650823 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.676358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1411/2822] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:22.680401 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.690883 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c597-756e-4241-8cc1-bcef250869ae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.747334 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.748002 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.748295 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.748571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.749252 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.751256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.770645 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1412/2823] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 15:36:22.774693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.785416 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c210e9-d960-4a7a-8253-741ccfa416a0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.789937 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.790599 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.790884 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.791165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.791917 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.793484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.803810 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1412/2824] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:22.807633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.817397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feebc98-3303-480b-aa7c-23383d691bf6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.821979 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.822850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.823138 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.823454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] PATH_INFO: `/v3/projects/4437916c37734f1795b3c6e43f9c6029/users/f32d205a35b64851a869ec81d1613980/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.838127 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4437916c37734f1795b3c6e43f9c6029, user_id=f32d205a35b64851a869ec81d1613980, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.840778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.853917 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1413/2825] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:22 2022] PUT /identity/v3/projects/4437916c37734f1795b3c6e43f9c6029/users/f32d205a35b64851a869ec81d1613980/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:22.857731 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.867820 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f75b55-d9e9-479d-ae12-1a1d636ec997 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.886800 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:22.924238 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.924905 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.925187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.925463 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.926184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.927798 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4378446-73dc-4631-8cae-5c51f96eb8a8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.937489 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1413/2826] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:22.941654 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:22.947091 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:22.953748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2128e6-785c-4106-a761-85bf21b832b4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:22.958976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:22.959897 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.960182 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.960460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] PATH_INFO: `/v3/projects/4437916c37734f1795b3c6e43f9c6029/users/f32d205a35b64851a869ec81d1613980/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:22.962423 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4437916c37734f1795b3c6e43f9c6029, user_id=f32d205a35b64851a869ec81d1613980, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:22.965071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:22.975551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1414/2827] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:22 2022] PUT /identity/v3/projects/4437916c37734f1795b3c6e43f9c6029/users/f32d205a35b64851a869ec81d1613980/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:22.985417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:22.985713 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:22.986002 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.011591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] MFA Rules not processed for user `f32d205a35b64851a869ec81d1613980`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:23.012807 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:23.079075 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1414/2828] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:23.090434 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:23.092272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:23.150015 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b384ff9-7f38-4766-95b4-2f5bbdcc6ba8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:23.153546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:23.154185 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.154484 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.154750 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.163581 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:23.165004 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4772f-e2a2-4aef-825e-e9ca92ed9416 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:23.174260 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1415/2829] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:23.201684 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:23.262881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a91f1-3aab-41b4-94e2-c723b8be02d5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:23.267753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:23.269250 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.269536 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.269813 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] PATH_INFO: `/v3/users/a4d1fed6ddd84dc2907336cc18adea1b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.271017 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d1fed6ddd84dc2907336cc18adea1b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:23.272993 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:23.318381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d1fed6ddd84dc2907336cc18adea1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.319561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:23.331965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d1fed6ddd84dc2907336cc18adea1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.338983 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d1fed6ddd84dc2907336cc18adea1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.346084 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d1fed6ddd84dc2907336cc18adea1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.346742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d1fed6ddd84dc2907336cc18adea1b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.352677 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1415/2830] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:23 2022] DELETE /identity/v3/users/a4d1fed6ddd84dc2907336cc18adea1b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:23.357142 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.357429 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.357711 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.381702 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:23.382902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:23.440372 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1416/2831] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:23.449784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:23.451165 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:23.507081 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96650733-d956-4024-a8c6-c0c44e85f6e7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:23.510643 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:23.511314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.511598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.511876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.520934 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:23.522349 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:23.532291 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1416/2832] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:23.794552 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:23.796067 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:23.808567 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ef5ba-2b61-4a34-a3a7-9dabf5e93bca None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:23.813171 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:23.814037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.814328 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.814691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] PATH_INFO: `/v3/projects/29fc48878a414050b14d0ddfd451cb5a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.816413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fc48878a414050b14d0ddfd451cb5a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:23.818316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:23.866369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fc48878a414050b14d0ddfd451cb5a', 'request_id': 'req-b7898b01-b7ee-4cbb-a119-c89af336aa98'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.873239 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fc48878a414050b14d0ddfd451cb5a', 'request_id': 'req-b7898b01-b7ee-4cbb-a119-c89af336aa98'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.879435 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fc48878a414050b14d0ddfd451cb5a', 'request_id': 'req-b7898b01-b7ee-4cbb-a119-c89af336aa98'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.889651 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] The token cache is being invalidate because project 29fc48878a414050b14d0ddfd451cb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:23.889939 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.890564 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:23.892120 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1417/2833] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:23 2022] DELETE /identity/v3/projects/29fc48878a414050b14d0ddfd451cb5a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:23.896301 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:23.905696 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff0e3d-ee16-403d-ba5d-f93eb4424a6d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:23.960316 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:23.961184 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:23.961467 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:23.961742 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] PATH_INFO: `/v3/users/f32d205a35b64851a869ec81d1613980` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:23.962910 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32d205a35b64851a869ec81d1613980)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:23.964829 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.011803 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32d205a35b64851a869ec81d1613980'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.023658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32d205a35b64851a869ec81d1613980'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.027302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:24.031256 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32d205a35b64851a869ec81d1613980'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.040332 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32d205a35b64851a869ec81d1613980'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.040965 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-017e4514-b1ab-4a1c-a3a8-42a38d07e06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32d205a35b64851a869ec81d1613980'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.046857 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1417/2834] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:23 2022] DELETE /identity/v3/users/f32d205a35b64851a869ec81d1613980 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:24.326592 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.328181 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:24.341682 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7898b01-b7ee-4cbb-a119-c89af336aa98 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.354129 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:24.401306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.402175 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.402463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.402744 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] PATH_INFO: `/v3/projects/4437916c37734f1795b3c6e43f9c6029` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.403960 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4437916c37734f1795b3c6e43f9c6029)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.405858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.449909 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4437916c37734f1795b3c6e43f9c6029', 'request_id': 'req-6acc6297-d026-47d4-be46-1449761df3f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.457331 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4437916c37734f1795b3c6e43f9c6029', 'request_id': 'req-6acc6297-d026-47d4-be46-1449761df3f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.463381 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4437916c37734f1795b3c6e43f9c6029', 'request_id': 'req-6acc6297-d026-47d4-be46-1449761df3f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.473936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] The token cache is being invalidate because project 4437916c37734f1795b3c6e43f9c6029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:24.474228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.474858 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6acc6297-d026-47d4-be46-1449761df3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:24.476399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1418/2835] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:24 2022] DELETE /identity/v3/projects/4437916c37734f1795b3c6e43f9c6029 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:24.481392 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.481710 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.481995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.499791 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:24.501016 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:24.559964 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1418/2836] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:36:24.568366 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.568661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.568969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.586737 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:24.587971 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:24.596301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1419/2837] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:24.600461 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.610276 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be287d48-6538-4cd2-8c22-453d55219022 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.615647 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.616320 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.616607 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.616891 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.617698 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:36:24.618013 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.619653 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.631294 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1419/2838] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:36:24.635566 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.645172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14d467-618f-4f34-9d79-692249323c57 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.649894 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.650546 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.650834 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.651113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.651841 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.653735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.679462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1420/2839] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:24 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:24.683525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.693328 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a469-d044-4f81-972f-7ccf617c8b27 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.748648 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.749311 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.749595 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.749927 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.750560 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.752506 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.773053 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1420/2840] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:24 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:24.777141 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.787816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069ff11-b1e2-4d26-8d9e-cb72a008fcdd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.792307 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.792963 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.793247 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.793522 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.794243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.795863 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.806597 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1421/2841] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:24.810347 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.820200 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7b3fa-ab24-4ec7-aaec-692850b75313 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.824752 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.825411 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.825697 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.825976 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.826704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.828303 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f171ce-4cd6-4f07-890f-58cd22608cca admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.838174 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1421/2842] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:24.841742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.852172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5690d-da8f-43e9-b5c0-49e39f59e4fa admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:24.856779 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:24.857659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.857948 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.858229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] PATH_INFO: `/v3/projects/52061acbc935404199bccbe0ebd8f6e2/users/e7472b8641644e5e82b29ac03be7b2a9/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.873605 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52061acbc935404199bccbe0ebd8f6e2, user_id=e7472b8641644e5e82b29ac03be7b2a9, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:24.876316 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:24.888880 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1422/2843] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:24 2022] PUT /identity/v3/projects/52061acbc935404199bccbe0ebd8f6e2/users/e7472b8641644e5e82b29ac03be7b2a9/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:24.894076 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212318-f190-463a-a011-b0c3fd9942ff None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:24.894375 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212318-f190-463a-a011-b0c3fd9942ff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:24.894663 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212318-f190-463a-a011-b0c3fd9942ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:24.912389 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dd212318-f190-463a-a011-b0c3fd9942ff None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:24.913582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dd212318-f190-463a-a011-b0c3fd9942ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:24.958792 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:24.972061 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1422/2844] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:24.981760 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:24.983481 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:25.030572 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:25.044691 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc32f6-e2e7-4516-b2e0-800334faf2e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:25.048355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:25.049034 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:25.049327 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:25.049611 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:25.058736 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:25.060898 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:25.070535 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1423/2845] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:25.870714 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:25.871095 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:25.871473 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:25.898933 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] MFA Rules not processed for user `e7472b8641644e5e82b29ac03be7b2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:25.900246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:25.968230 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1423/2846] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 15:36:25.972690 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:25.982292 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25d9ed-4b97-4613-aa6b-6fc68b83a4ca service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:25.987397 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:25.988058 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:25.988344 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:25.988620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:25.989306 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:25.991192 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.017267 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1424/2847] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:25 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:26.021608 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.032483 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c3467-29e4-485d-9039-68bffa9f2aea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:26.088070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:26.088745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.089031 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.089306 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.090006 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:26.091949 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.111269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1424/2848] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:26 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:26.115827 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.126565 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556f826-0eb9-4a5e-a7a0-8d9e62a4a38f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:26.131166 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:26.131859 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.132146 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.132454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.133208 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:26.134808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.145355 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1425/2849] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:26.149340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.159612 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054b21-6aeb-4f0c-b2ed-27d6eb3b1e35 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:26.164125 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:26.165001 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.165283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.165561 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] PATH_INFO: `/v3/projects/ab84df02408c4f2086ef531eee0b631a/users/05fce312509f4ae4b8b7755d6b8d6967/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.181693 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab84df02408c4f2086ef531eee0b631a, user_id=05fce312509f4ae4b8b7755d6b8d6967, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:26.184508 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.197033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1425/2850] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:26 2022] PUT /identity/v3/projects/ab84df02408c4f2086ef531eee0b631a/users/05fce312509f4ae4b8b7755d6b8d6967/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:26.200854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.210629 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac174a6-2dee-4399-9998-c969ccd97246 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:26.265715 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:26.266399 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.266689 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.266969 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.268082 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:26.269638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c4e84-ce31-44cb-bad9-e61ead9922f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.279269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1426/2851] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:26.284037 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.294434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88738542-e95b-4b92-9288-f24b4de15d97 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:26.299581 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:26.300470 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.300754 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.301033 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] PATH_INFO: `/v3/projects/ab84df02408c4f2086ef531eee0b631a/users/05fce312509f4ae4b8b7755d6b8d6967/roles/84f17f17b9694564b251da9fa908f805` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.303012 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab84df02408c4f2086ef531eee0b631a, user_id=05fce312509f4ae4b8b7755d6b8d6967, role_id=84f17f17b9694564b251da9fa908f805)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:26.305667 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:26.315722 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1426/2852] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:26 2022] PUT /identity/v3/projects/ab84df02408c4f2086ef531eee0b631a/users/05fce312509f4ae4b8b7755d6b8d6967/roles/84f17f17b9694564b251da9fa908f805 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:36:26.887822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:26.888174 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:26.888462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:26.915232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] MFA Rules not processed for user `05fce312509f4ae4b8b7755d6b8d6967`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:26.916491 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:26.986879 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1427/2853] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:26.997783 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:26.999652 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:27.042233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:27.060977 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e8c0e-5244-4ab3-8a82-4f3fd23a05ea admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:27.065103 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:27.065757 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:27.066039 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:27.066314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:27.124510 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:27.125881 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:27.135521 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1427/2854] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:28.466854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:28.468272 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:28.480498 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba742f6-aba9-492e-853d-272742d8a7db None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:28.484054 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:28.484696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:28.484980 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:28.485258 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:28.494464 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:28.495932 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:28.505709 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1428/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:37.131532 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=32 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:38.501083 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:41.190694 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:41.192209 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:41.244833 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:41.271665 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903617c8-4bcf-4b05-bb7c-f328effc608b service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:41.276704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:41.277360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:41.277645 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:41.277926 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:41.279876 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:41.293118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:41.294828 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:41.305796 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1428/2856] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:41.311878 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efd1c-7ed6-43f4-a1c9-5f8c4b87429f service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:41.316520 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:41.317402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:41.317692 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:41.317976 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] PATH_INFO: `/v3/users/e7472b8641644e5e82b29ac03be7b2a9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:41.319211 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7472b8641644e5e82b29ac03be7b2a9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:41.321128 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:41.367641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7472b8641644e5e82b29ac03be7b2a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.368296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7472b8641644e5e82b29ac03be7b2a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.375110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7472b8641644e5e82b29ac03be7b2a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.387237 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7472b8641644e5e82b29ac03be7b2a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.394218 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7472b8641644e5e82b29ac03be7b2a9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.400263 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1429/2857] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:41 2022] DELETE /identity/v3/users/e7472b8641644e5e82b29ac03be7b2a9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:41.573744 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:41.584996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0357b1-27d0-4138-a0ed-2c29ad0363ce service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:41.644272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:41.645145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:41.645431 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:41.645712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] PATH_INFO: `/v3/projects/52061acbc935404199bccbe0ebd8f6e2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:41.646894 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52061acbc935404199bccbe0ebd8f6e2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:41.648816 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:41.695426 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52061acbc935404199bccbe0ebd8f6e2', 'request_id': 'req-c667d9b2-2c31-4008-b502-3dae183afab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.701330 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52061acbc935404199bccbe0ebd8f6e2', 'request_id': 'req-c667d9b2-2c31-4008-b502-3dae183afab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.707512 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52061acbc935404199bccbe0ebd8f6e2', 'request_id': 'req-c667d9b2-2c31-4008-b502-3dae183afab7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.719099 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] The token cache is being invalidate because project 52061acbc935404199bccbe0ebd8f6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:41.719417 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.720054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.722171 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1429/2858] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:41 2022] DELETE /identity/v3/projects/52061acbc935404199bccbe0ebd8f6e2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:36:41.726161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:41.727652 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:41.740502 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f58320-584a-482a-a9b4-58e54b4272b8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:41.795006 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:41.795910 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:41.796193 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:41.796465 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] PATH_INFO: `/v3/users/05fce312509f4ae4b8b7755d6b8d6967` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:41.797637 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05fce312509f4ae4b8b7755d6b8d6967)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:41.799610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:41.821337 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:41.845420 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05fce312509f4ae4b8b7755d6b8d6967'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.846699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05fce312509f4ae4b8b7755d6b8d6967'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.853900 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05fce312509f4ae4b8b7755d6b8d6967'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.866301 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05fce312509f4ae4b8b7755d6b8d6967'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.872937 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1c5d0e3a-0b13-40c3-8974-5e10e5ffbe45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05fce312509f4ae4b8b7755d6b8d6967'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:41.879244 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1430/2859] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:41 2022] DELETE /identity/v3/users/05fce312509f4ae4b8b7755d6b8d6967 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:42.047360 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.048786 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:42.063716 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667d9b2-2c31-4008-b502-3dae183afab7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.082779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:42.125118 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.125998 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.126289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.126597 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] PATH_INFO: `/v3/projects/ab84df02408c4f2086ef531eee0b631a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.128319 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab84df02408c4f2086ef531eee0b631a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.130213 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.177656 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab84df02408c4f2086ef531eee0b631a', 'request_id': 'req-8a454115-8139-4d53-b28d-6c147251b3c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.183882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab84df02408c4f2086ef531eee0b631a', 'request_id': 'req-8a454115-8139-4d53-b28d-6c147251b3c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.190222 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab84df02408c4f2086ef531eee0b631a', 'request_id': 'req-8a454115-8139-4d53-b28d-6c147251b3c1'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.201538 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] The token cache is being invalidate because project ab84df02408c4f2086ef531eee0b631a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:42.202121 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.202434 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8a454115-8139-4d53-b28d-6c147251b3c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.204007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1430/2860] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:42 2022] DELETE /identity/v3/projects/ab84df02408c4f2086ef531eee0b631a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:42.209463 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.209757 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.210039 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.227787 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:42.228930 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:42.287161 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1431/2861] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:42.295772 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.296085 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.296344 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.315352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:42.316551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:42.324368 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1431/2862] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:42.328377 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.338087 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22090d8-c2de-40e9-851e-ab6730c13cc0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.342661 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.343350 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.343642 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.343923 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.344659 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:36:42.344977 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.346620 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.358180 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1432/2863] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:36:42.362611 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.371709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a24ba-ceb0-4d0a-8f7a-b89f2e7d686b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.376246 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.376889 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.377794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.378072 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.378753 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.380658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.404931 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1432/2864] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:42.408926 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.418726 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5ee41-88c8-4940-99e1-d1804d92177e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.476332 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.477013 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.477296 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.477571 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.478314 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.480177 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.500523 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1433/2865] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:42.505282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.515440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615269-e41b-42fa-b094-7bc877a1c30a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.520553 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.521233 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.521522 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.521805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.522547 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.524215 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.534870 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1433/2866] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:42.538610 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.548556 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4da8ff-7abd-4f61-bda8-5b4ab2440667 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.553684 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.554349 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.554638 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.554921 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.555698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.557286 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7ee9-3c45-4b0e-996c-1b44be8ccc42 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.567489 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1434/2867] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:42.571322 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.581062 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254d59b-26cb-4d47-9f08-ed13f53b54c3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.585542 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.586433 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.586719 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.586996 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] PATH_INFO: `/v3/projects/2c2fcd0935fb4731943e30ef63a75326/users/87e24d163d6547b98423ddf285c3f8ec/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.603478 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2fcd0935fb4731943e30ef63a75326, user_id=87e24d163d6547b98423ddf285c3f8ec, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.606860 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.624132 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1434/2868] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:42 2022] PUT /identity/v3/projects/2c2fcd0935fb4731943e30ef63a75326/users/87e24d163d6547b98423ddf285c3f8ec/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:36:42.634379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.634676 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.634964 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.641724 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:42.662096 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] MFA Rules not processed for user `87e24d163d6547b98423ddf285c3f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:42.663333 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:42.725187 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1435/2869] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:42.736329 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.738745 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:42.789755 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:42.796289 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24bb0f-a2f1-4516-bbef-5f0434c952d8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.801158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.801845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.802314 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.802604 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.810902 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.812358 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913611a3-cb7a-4afe-8cd7-4d3e4e05d454 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.821073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1435/2870] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:42.840256 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:42.900573 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2325e8a-d422-4f5c-8d6e-c9fafc8e442a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:42.905102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:42.905957 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.906243 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.906518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] PATH_INFO: `/v3/users/87e24d163d6547b98423ddf285c3f8ec` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:42.907729 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e24d163d6547b98423ddf285c3f8ec)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:42.910115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:42.952578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e24d163d6547b98423ddf285c3f8ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.953220 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e24d163d6547b98423ddf285c3f8ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.960168 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e24d163d6547b98423ddf285c3f8ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.972159 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e24d163d6547b98423ddf285c3f8ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.978728 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e24d163d6547b98423ddf285c3f8ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:42.984501 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1436/2871] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:36:42 2022] DELETE /identity/v3/users/87e24d163d6547b98423ddf285c3f8ec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:42.989672 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:42.989960 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:42.990237 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.008257 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:43.009443 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.066761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1436/2872] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:43.076816 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.078228 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.092854 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:43.137615 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ee9c4-ed80-4af5-ba4a-4bc6bed3ceb3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.141454 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.142102 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.142386 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.143460 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.152680 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.154110 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16b759-ccb1-441e-86e9-b7867efe91a8 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.164739 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1437/2873] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:43.426268 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.427762 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.440073 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0855ea0-f929-4f25-b8f8-27ac52248dd3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.444636 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.445504 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.445794 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.446057 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] PATH_INFO: `/v3/projects/2c2fcd0935fb4731943e30ef63a75326` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.447366 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2fcd0935fb4731943e30ef63a75326)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.449340 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.496838 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2fcd0935fb4731943e30ef63a75326', 'request_id': 'req-554bbb35-e8df-4ce7-adcf-a9051dac78db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:43.503140 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2fcd0935fb4731943e30ef63a75326', 'request_id': 'req-554bbb35-e8df-4ce7-adcf-a9051dac78db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:43.508760 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2fcd0935fb4731943e30ef63a75326', 'request_id': 'req-554bbb35-e8df-4ce7-adcf-a9051dac78db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:43.520939 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] The token cache is being invalidate because project 2c2fcd0935fb4731943e30ef63a75326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:36:43.521231 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:43.521842 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-554bbb35-e8df-4ce7-adcf-a9051dac78db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:36:43.523385 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1437/2874] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:36:43 2022] DELETE /identity/v3/projects/2c2fcd0935fb4731943e30ef63a75326 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:43.528215 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.528696 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.528803 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.546895 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:43.548137 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.605962 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1438/2875] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:43.614120 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.614440 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.614724 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.632457 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:43.633637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.641283 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1438/2876] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:43.645505 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.655078 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a5990-3bd6-4b0f-b17e-adca49da2fbd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.660151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.660805 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.661092 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.661374 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.662101 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 15:36:43.662415 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.664041 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.676158 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1439/2877] 10.222.0.46 () {58 vars in 1152 bytes} [Wed Jul 6 15:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 15:36:43.681681 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.690981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bffa1-1b35-4b95-a7d0-96d6c5a22432 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.695858 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.696565 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.696849 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.697123 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.697808 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.699914 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.723981 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1439/2878] 10.222.0.46 () {60 vars in 1152 bytes} [Wed Jul 6 15:36:43 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:43.728209 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.738735 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065402f-4283-45c6-a98c-d93d4ccfc235 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.793500 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.794234 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.794521 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.794811 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.795526 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.797431 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.816961 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1440/2879] 10.222.0.46 () {60 vars in 1143 bytes} [Wed Jul 6 15:36:43 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 15:36:43.820453 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.831070 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a224e8a-b1f9-44b5-8e60-d94f99f20453 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.835640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.836302 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.836591 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.836873 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.837606 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.839187 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.850151 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1440/2880] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:43.853936 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.863815 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7890a-276a-4a47-8368-1fc61afe562f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.868402 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.869057 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.869347 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.869630 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.870359 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.871988 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc308831-d943-45fe-a56a-31ea8e652175 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.881639 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1441/2881] 10.222.0.46 () {58 vars in 1121 bytes} [Wed Jul 6 15:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 15:36:43.885381 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:43.895959 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fa2d-d507-4ebe-983d-c7c2a0906295 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:43.900546 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:43.901420 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.901705 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.902109 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] PATH_INFO: `/v3/projects/38d74ef2d26745779c4fc17d59a853eb/users/6696623b78bc46c6a1bfebca4f98d6fe/roles/86d14aa8b1d24e4587187044c5bbeb72` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.917154 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d74ef2d26745779c4fc17d59a853eb, user_id=6696623b78bc46c6a1bfebca4f98d6fe, role_id=86d14aa8b1d24e4587187044c5bbeb72)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:43.919804 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:43.932556 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1441/2882] 10.222.0.46 () {60 vars in 1482 bytes} [Wed Jul 6 15:36:43 2022] PUT /identity/v3/projects/38d74ef2d26745779c4fc17d59a853eb/users/6696623b78bc46c6a1bfebca4f98d6fe/roles/86d14aa8b1d24e4587187044c5bbeb72 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:36:43.943575 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be8d7-560c-471e-a22a-2289884975ed None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:43.943865 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be8d7-560c-471e-a22a-2289884975ed None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:43.944151 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272be8d7-560c-471e-a22a-2289884975ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:43.970299 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-272be8d7-560c-471e-a22a-2289884975ed None None] MFA Rules not processed for user `6696623b78bc46c6a1bfebca4f98d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:36:43.972148 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-272be8d7-560c-471e-a22a-2289884975ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:43.981105 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:44.039085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1442/2883] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:36:44.049715 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:36:44.051403 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:36:44.061980 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:44.110272 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801a0d9-cadb-464b-85f5-9465f2ce3eeb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:36:44.113929 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edbb85bda10d45bf8f66c6ee0561c2a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:36:44.114576 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:36:44.114864 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:36:44.115150 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:36:44.124273 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:36:44.125709 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649dc6f-fe56-4989-9c89-3a8b285d5b4d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:36:44.135145 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1442/2884] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:36:54.036225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=22 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:36:54.131054 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=22 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:00.197414 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272be8d7-560c-471e-a22a-2289884975ed None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:00.260390 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272be8d7-560c-471e-a22a-2289884975ed None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:00.264938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:00.265822 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:00.266108 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:00.266387 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] PATH_INFO: `/v3/users/6696623b78bc46c6a1bfebca4f98d6fe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:00.267612 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6696623b78bc46c6a1bfebca4f98d6fe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:00.269837 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:00.317115 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6696623b78bc46c6a1bfebca4f98d6fe'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.317808 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6696623b78bc46c6a1bfebca4f98d6fe'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.324433 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6696623b78bc46c6a1bfebca4f98d6fe'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.336119 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6696623b78bc46c6a1bfebca4f98d6fe'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.342902 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6696623b78bc46c6a1bfebca4f98d6fe'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.348844 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1443/2885] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:37:00 2022] DELETE /identity/v3/users/6696623b78bc46c6a1bfebca4f98d6fe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:37:00.353364 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:00.353844 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:00.353954 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:00.371551 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] MFA Rules not processed for user `d82646fafad44bd489ae80854728f595`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:37:00.372789 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:00.431142 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1443/2886] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:37:00.441354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:00.442778 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:00.497952 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e47e9-b867-4492-be12-aaf8eb191e8b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:00.501579 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:00.502225 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:00.502512 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:00.503413 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:00.512354 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:00.513773 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1226774-8a6a-497b-93e5-1e15124a6cad service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:00.523666 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1444/2887] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 15:37:00.783127 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:00.785083 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:00.798885 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc3c72-dc79-4796-a38b-35abd4af246d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:00.804219 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:00.805919 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:00.806210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:00.806484 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] PATH_INFO: `/v3/projects/38d74ef2d26745779c4fc17d59a853eb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:00.807882 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d74ef2d26745779c4fc17d59a853eb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:00.810134 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:00.855621 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d74ef2d26745779c4fc17d59a853eb', 'request_id': 'req-68e2b054-2385-46d5-ba9c-3e836bd03335'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.861658 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d74ef2d26745779c4fc17d59a853eb', 'request_id': 'req-68e2b054-2385-46d5-ba9c-3e836bd03335'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.867944 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d74ef2d26745779c4fc17d59a853eb', 'request_id': 'req-68e2b054-2385-46d5-ba9c-3e836bd03335'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.879778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] The token cache is being invalidate because project 38d74ef2d26745779c4fc17d59a853eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:37:00.880071 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.880685 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:37:00.882210 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1444/2888] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:37:00 2022] DELETE /identity/v3/projects/38d74ef2d26745779c4fc17d59a853eb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:37:10.520068 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:10.876805 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:19.820831 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:19.821326 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:19.821448 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:19.838800 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] MFA Rules not processed for user `7f9e0665a37d490f8b62205c5890da56`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 15:37:19.840050 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:19.894755 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1445/2889] 10.222.0.46 () {58 vars in 957 bytes} [Wed Jul 6 15:37:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 15:37:20.208750 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:20.210180 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:20.266778 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2b054-2385-46d5-ba9c-3e836bd03335 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:20.270610 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:20.271282 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:20.271572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:20.271852 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:20.273640 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:20.282956 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:20.284400 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:20.292936 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1445/2890] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:37:20.726232 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:20.727698 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:20.798628 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3276a-991d-4f4f-ada5-5f70eb319828 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:20.803578 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:20.804229 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:20.804518 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:20.804797 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:20.806588 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:20.818456 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:20.820071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:20.830298 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1446/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:37:30.289304 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=26 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:30.826172 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=57 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:33.671887 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:33.673357 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:33.798639 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f7bcd-aad0-4c30-ac9c-934196af5e4e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:33.804334 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:33.804995 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:33.805872 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:33.806158 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:33.807987 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:33.820951 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:33.822525 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:33.833945 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1446/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:37:37.911147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:37.912646 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:37.978071 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7e4c5-05b7-431e-b099-f3cafb302534 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:37.983776 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aff09cb4b1ff42c0a011f8e09a3c6c38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:37.984437 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:37.984727 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:37.985010 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:37.986794 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:37.999266 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:38.000911 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:38.011742 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1447/2893] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:37:43.829415 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=56 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:48.007124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=28 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:37:53.665269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:37:53.686492 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bd15e-f36a-484a-b63b-6ef6e4b2f03e service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:37:53.691543 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:37:53.692191 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:37:53.692479 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:37:53.692761 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:37:53.694499 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:37:53.717779 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:37:53.719393 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:37:53.729841 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1447/2894] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:38:03.725623 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=15 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:39:06.781641 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:06.801063 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cdced-7f6c-4841-93c7-4308be8895b5 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:06.804633 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:06.805280 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:06.805568 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:06.805848 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:06.807653 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:39:06.818824 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:06.820379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:06.829446 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1448/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:39:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:39:16.825479 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=10 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:39:20.000263 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:20.009633 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c33c0-17e2-41b4-b2cc-07dd155c4ea9 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:20.013704 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c6dce92d6724fbf878faba138471bb8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:20.014342 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:20.014631 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:20.014907 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:20.016784 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:39:20.102116 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:20.103598 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:20.122494 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1448/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:39:26.244152 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:26.246190 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:39:26.262428 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5d185-9e07-45bb-9ed6-ffa1bedcb1e3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:26.267226 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:26.268373 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:26.268658 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:26.268938 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] PATH_INFO: `/v3/users/7f9e0665a37d490f8b62205c5890da56` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:26.270122 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9e0665a37d490f8b62205c5890da56)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:26.272113 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:26.317059 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9e0665a37d490f8b62205c5890da56'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.317699 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9e0665a37d490f8b62205c5890da56'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.324595 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9e0665a37d490f8b62205c5890da56'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.336596 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9e0665a37d490f8b62205c5890da56'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.343818 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9e0665a37d490f8b62205c5890da56'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.349551 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1449/2897] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:39:26 2022] DELETE /identity/v3/users/7f9e0665a37d490f8b62205c5890da56 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:39:26.521614 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:26.532359 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954d9d7-f959-49d6-9769-46143c93f4db service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:26.591424 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:26.592293 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:26.592572 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:26.592845 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] PATH_INFO: `/v3/projects/6d724d364f9b4bafb241a0bb0b02429f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:26.594007 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d724d364f9b4bafb241a0bb0b02429f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:26.595940 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:26.643281 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d724d364f9b4bafb241a0bb0b02429f', 'request_id': 'req-cc5cfb03-9a15-4150-b30d-faad1f3a0973'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.649582 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d724d364f9b4bafb241a0bb0b02429f', 'request_id': 'req-cc5cfb03-9a15-4150-b30d-faad1f3a0973'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.655961 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d724d364f9b4bafb241a0bb0b02429f', 'request_id': 'req-cc5cfb03-9a15-4150-b30d-faad1f3a0973'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.659583 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:39:26.672613 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] The token cache is being invalidate because project 6d724d364f9b4bafb241a0bb0b02429f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:39:26.672909 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.673555 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.675527 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1449/2898] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:39:26 2022] DELETE /identity/v3/projects/6d724d364f9b4bafb241a0bb0b02429f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:39:26.679085 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:26.680517 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:39:26.693369 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811066c-706b-4524-a696-776235b941b7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:26.750269 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:26.751147 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:26.751474 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:26.751751 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] PATH_INFO: `/v3/users/34c23d5790054071b9326be4a7c3036b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:26.752935 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c23d5790054071b9326be4a7c3036b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:26.754850 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:26.799462 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c23d5790054071b9326be4a7c3036b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.800124 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c23d5790054071b9326be4a7c3036b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.808408 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c23d5790054071b9326be4a7c3036b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.819906 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c23d5790054071b9326be4a7c3036b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.821826 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:39:26.828430 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c23d5790054071b9326be4a7c3036b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:26.835061 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1450/2899] 10.222.0.46 () {58 vars in 1223 bytes} [Wed Jul 6 15:39:26 2022] DELETE /identity/v3/users/34c23d5790054071b9326be4a7c3036b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 15:39:27.007839 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:39:27.009269 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:39:27.024637 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cfb03-9a15-4150-b30d-faad1f3a0973 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:39:27.081066 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd82646fafad44bd489ae80854728f595', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b898c8312f89416b8b105c391af48831', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:39:27.081968 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:39:27.082259 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:39:27.082544 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] PATH_INFO: `/v3/projects/260c02062b8247ee8adff900243d628a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:39:27.083788 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=260c02062b8247ee8adff900243d628a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:39:27.085712 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:39:27.130352 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '260c02062b8247ee8adff900243d628a', 'request_id': 'req-05c9051e-37aa-4b65-91c5-190c488ca047'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:27.137195 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '260c02062b8247ee8adff900243d628a', 'request_id': 'req-05c9051e-37aa-4b65-91c5-190c488ca047'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:27.142630 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '260c02062b8247ee8adff900243d628a', 'request_id': 'req-05c9051e-37aa-4b65-91c5-190c488ca047'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:27.154447 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] The token cache is being invalidate because project 260c02062b8247ee8adff900243d628a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 15:39:27.154737 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:27.155386 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-05c9051e-37aa-4b65-91c5-190c488ca047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 15:39:27.156932 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1450/2900] 10.222.0.46 () {58 vars in 1232 bytes} [Wed Jul 6 15:39:27 2022] DELETE /identity/v3/projects/260c02062b8247ee8adff900243d628a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 15:39:36.832003 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:39:37.151545 dv-cinder-848848-4-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 15:43:01.641056 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 15:43:01.642748 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:43:01.711037 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d5fae-b390-4136-8b7d-08c93a1d51d8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 15:43:01.767468 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '319c09d5d4db4a18bf1191375252c541', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0da64de84a214a5bbc0d2c0105cc892a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 15:43:01.768112 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 15:43:01.768393 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 15:43:01.769195 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 15:43:01.770945 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 15:43:01.781777 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 15:43:01.783379 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c21dc-ae66-412f-af1d-c31f1a55e0c0 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 15:43:01.793710 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1451/2901] 10.222.0.46 () {60 vars in 1365 bytes} [Wed Jul 6 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 15:43:11.789533 dv-cinder-848848-4-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=58 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}